-- Logs begin at Mon 2022-05-09 17:33:35 UTC, end at Mon 2022-05-09 18:49:40 UTC. -- May 09 18:15:03.277126 dv-cinder-841125-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 09 18:15:03.285805 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 09 18:15:03.288592 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 9 18:15:03 2022] *** May 09 18:15:03.288592 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 09 18:15:03.288592 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 09 18:15:03.288592 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: nodename: dv-cinder-841125-3-iscsi May 09 18:15:03.288592 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: machine: x86_64 May 09 18:15:03.288592 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: clock source: unix May 09 18:15:03.288592 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: pcre jit disabled May 09 18:15:03.288592 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: detected number of CPU cores: 4 May 09 18:15:03.288592 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: current working directory: / May 09 18:15:03.288592 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: detected binary path: /usr/bin/uwsgi-core May 09 18:15:03.288592 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: your processes number limit is 47680 May 09 18:15:03.289318 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: your memory page size is 4096 bytes May 09 18:15:03.289318 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: detected max file descriptor number: 2048 May 09 18:15:03.289318 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: lock engine: pthread robust mutexes May 09 18:15:03.289318 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: thunder lock: enabled May 09 18:15:03.289318 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 09 18:15:03.289318 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 09 18:15:03.334228 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: Python main interpreter initialized at 0x55b3820469c0 May 09 18:15:03.334228 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: python threads support enabled May 09 18:15:03.334228 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: your server socket listen backlog is limited to 100 connections May 09 18:15:03.334228 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: your mercy for graceful operations on workers is 90 seconds May 09 18:15:03.334655 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: mapped 403077 bytes (393 KB) for 2 cores May 09 18:15:03.334744 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: *** Operational MODE: preforking *** May 09 18:15:03.335034 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: *** uWSGI is running in multiple interpreter mode *** May 09 18:15:03.335034 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: spawned uWSGI master process (pid: 79815) May 09 18:15:03.335267 dv-cinder-841125-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 09 18:15:03.338077 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: spawned uWSGI worker 1 (pid: 79825, cores: 1) May 09 18:15:03.338077 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: spawned uWSGI worker 2 (pid: 79826, cores: 1) May 09 18:15:03.338077 dv-cinder-841125-3-iscsi devstack@keystone.service[79815]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 09 18:15:09.023879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 18:15:09.023879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: from cryptography.utils import int_from_bytes May 09 18:15:09.025173 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 18:15:09.025173 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: from cryptography.utils import int_from_bytes May 09 18:15:09.025312 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 18:15:09.025312 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: from cryptography.utils import int_from_bytes May 09 18:15:09.026625 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 18:15:09.026625 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: from cryptography.utils import int_from_bytes May 09 18:15:09.074474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 18:15:09.074658 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 18:15:09.074877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] command line args: [] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 18:15:09.075099 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 18:15:09.075354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ================================================================================ {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 18:15:09.075547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] admin_token = **** {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.075628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 18:15:09.075792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] config_dir = [] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.075882 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 18:15:09.076018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.076104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] command line args: [] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 18:15:09.076173 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] config_source = [] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.076238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 18:15:09.076303 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] debug = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.076469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ================================================================================ {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 18:15:09.076554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.078796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] admin_token = **** {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.078871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.078931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] config_dir = [] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.078988 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] insecure_debug = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.079050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.079050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] config_source = [] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.079169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.079483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] debug = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.079549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.079613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.079707 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] list_limit = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.079769 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.079831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] log_config_append = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.079889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] insecure_debug = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.079951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.079951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] log_dir = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080068 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] log_file = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] list_limit = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] log_config_append = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080542 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] log_dir = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] log_file = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080783 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080905 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.080969 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081027 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081084 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081278 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081398 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081455 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081722 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081786 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] notification_format = cadf {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.081979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.082049 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.082116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.082116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.082229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.082293 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.082355 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] publish_errors = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.082418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] notification_format = cadf {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.082485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.082548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.082611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.083440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.083440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.084048 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.084404 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.084744 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] publish_errors = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.085069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] standard_threads = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.085367 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.085863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] strict_password_check = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.086143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.086313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.086509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.086701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] use_eventlog = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.086701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] use_journal = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.087088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.087338 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] use_json = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.087548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.087811 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] use_stderr = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.088147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] standard_threads = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.088504 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] use_syslog = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.088699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] strict_password_check = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.088863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] watch_log_file = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.089013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.089013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] use_eventlog = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.089286 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.089434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] use_journal = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.089563 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.089740 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] use_json = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.089740 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] use_stderr = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.090111 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.090330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] use_syslog = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.090554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.090774 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] watch_log_file = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 18:15:09.090774 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.091249 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.091460 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.091460 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.091886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.092109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.092109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.092538 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.092762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.093088 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.093334 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.093555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.093765 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.093765 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.094153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.094381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.094381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.094776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.094985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.095231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.095456 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.095456 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.095882 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.096099 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.096328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.096540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.096540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.096964 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.097181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.097397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.097620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.097834 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.098049 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.098049 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.098462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.099256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.099488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.099719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.099719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.100152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.100368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.100368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.100769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.100975 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.101188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.101417 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.101632 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.101852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.102067 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.102282 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.102506 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.102752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.102979 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.103116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.103279 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.103409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.103599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.103931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.104083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.104213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.104342 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.104476 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.104604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.104741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.104881 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.105009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.105136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.105262 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.105402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.105531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.105657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.105784 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.105910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.106037 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.106175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.106304 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.106438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.106576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.106703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.106836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.106978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.107111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.107262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.107391 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.107518 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.107645 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.107783 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.107911 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.108038 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.108165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.108292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.109613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.109824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.109992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.110217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.110383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.110576 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.110768 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.110994 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.111162 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.111451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.111451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.111822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.111822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.112230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.112230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.112569 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.112758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.112947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.113136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.113360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.113526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.113718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.113909 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.114097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.114900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.115105 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.115292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.115470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.115614 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.115745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.115907 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.116039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.116176 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.116306 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.116442 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.116602 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.116744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.116875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.116875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.117142 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.117282 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.117418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.117552 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.117552 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.117793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.117961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.118095 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.118229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.118358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.118486 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.enabled = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.118622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.118760 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.118888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.119028 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.119155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.119319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.119458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.119610 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.119748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.119889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.120025 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.120160 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.120369 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.enabled = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.120507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.120707 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.120854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.120989 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.121135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.121286 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.121423 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.121557 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.121690 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.121824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.121958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.122101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.122237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.122370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.122504 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.122637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.122772 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.122913 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.123059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.123220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.123363 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.123497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.123632 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.123777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.123926 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.124072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.124210 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.124344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.124478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.124611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.124745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.124887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.125033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.125168 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.125310 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.125451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.125586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.125733 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.125869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.126003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.126137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.126271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.126406 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.126552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.126688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] auth.external = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.126832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.126973 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] auth.mapped = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.127115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.127276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.127677 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.127820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.127954 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.128089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] auth.password = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.128222 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.128369 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] auth.token = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.128536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.128681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.128829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] auth.external = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.128966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] catalog.caching = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.129111 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] auth.mapped = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.129291 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.129433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.129582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.129726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.129726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] auth.password = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.130013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.130156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] auth.token = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.130293 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.130437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.130574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.130716 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] catalog.caching = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.130853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] credential.caching = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.131006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.131151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] credential.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.131358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.131498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.131635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.131913 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.132057 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.132197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.132344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.132481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.132618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] credential.caching = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.132759 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.132900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] credential.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.133036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.133191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.133329 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.133464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.133599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.133742 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.133887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.134031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.134031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.134287 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.134422 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.134556 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.134697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.134841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.134988 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.135123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.135292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.135430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.135430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] federation.caching = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.135696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.135834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.135969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.136104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] federation.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.136237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.136372 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.136515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.136651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.136835 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.136997 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.137144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] federation.caching = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.137288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.137500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.137641 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.137776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] federation.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.137912 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.138046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.138046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.138309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.138309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.138565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.138565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.138836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.138836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity.caching = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.139100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.139100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.139421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.139556 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.139690 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.139825 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.139973 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.140116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.140252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.140388 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity.caching = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.140522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.140656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.140812 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.140949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.141085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.141219 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.141354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.141488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.141631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.141767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.141902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.142037 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.142172 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.142748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.142901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.143050 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.143250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.143408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.143547 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.143694 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.143832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.143967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.144102 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.144238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.144374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.144517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.144654 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.144789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.144924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.145069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.145206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.145349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.145516 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.145653 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.145788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.145923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.146058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.146201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.146340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.146476 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.146611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.146746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.146891 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.147035 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.147207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.147352 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.147487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.147621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.147756 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.147902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.148039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.148175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.148317 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.148454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.148589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.148732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.148732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.149000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.149134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.149269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.149404 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.149553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.149691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.149827 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.password = **** {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.149961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.150096 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.150230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.150373 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.150510 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.150645 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.150791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.password = **** {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.150925 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.151059 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.151220 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.151363 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.151500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.151636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.151771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.151906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.152056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.152193 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.152328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.152462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.152608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.152744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.152887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.153024 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.153158 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.153292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.153427 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.153562 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.153707 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.153843 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.153978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.154112 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.154247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.154382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.154535 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.154673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.154809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.154944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.155079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.155265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.155420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.155558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.155695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.155829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.155970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.156106 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.156250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.156386 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.156531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.156666 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.156802 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.156944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.157087 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.157227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.157363 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.157497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.157631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.157776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.157913 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.158047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.158181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.158325 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.158460 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.158602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.158738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.158874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.159009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.159143 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.159302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.159451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.159588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.159723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.159859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.159992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.160127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.160279 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.160416 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.160550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.160686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.160820 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.160956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.161612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.161751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.161886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.162021 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] policy.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.162156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.162291 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.162434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.162571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.162715 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.162851 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.162986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] policy.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.163122 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] receipt.caching = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.163337 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.163476 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.163611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.163746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.163881 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.164016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.164162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] receipt.caching = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.164298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.164434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.164579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.164714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.164849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] resource.caching = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.165058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.165058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.165319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.165454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.165588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] resource.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.165722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] resource.caching = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.165865 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.166001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.166136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.166272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] resource.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.166406 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.166554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.166699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] revoke.caching = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.166835 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.166970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.167104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.167259 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.167397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] revoke.caching = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.167544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] role.cache_time = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.167691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.167827 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] role.caching = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.168034 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.168187 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] role.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.168325 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] role.cache_time = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.168471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] role.list_limit = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.168619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] role.caching = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.168759 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.168894 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] role.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.169030 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.169164 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] role.list_limit = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.169309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.169446 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.169580 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.169715 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.169850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.169986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.170134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.170271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.170417 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.170559 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.170702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.170837 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.170980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.171116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.171291 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.171429 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.171564 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.171711 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.171847 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.171983 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.172117 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.172262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.172262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.172529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.172666 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.172800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.172800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.173061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.173197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.173341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.173478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.173612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.173748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.173882 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.174026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.174170 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.174307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.174449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.174449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.174708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.174708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.174971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.175108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.175277 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.175415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.175549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.175685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.175831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.175831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.176099 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.176233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.176368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.176502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.176670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.177335 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.177475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.177616 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.177757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.177891 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.178035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] token.cache_time = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.178172 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.178317 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] token.caching = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.178453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] token.cache_time = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.178588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.178732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] token.caching = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.178869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] token.provider = fernet {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.179003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.179138 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.179306 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] token.provider = fernet {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.179444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.179591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.179728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.179865 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.180000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.180134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.180279 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.180423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.180560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.180695 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.180831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] trust.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.180966 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.181101 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.181244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] trust.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.181381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.181516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.181652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.181787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.181923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.182071 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.182220 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.182356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.182490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.182625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.182760 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.182904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.183042 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.183199 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.183343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.connection = **** {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.183492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.183640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.183777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.connection = **** {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.183922 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.184079 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.184225 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.184362 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.184497 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.184632 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.184768 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.184902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.185046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.185184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.185325 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.185461 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.185598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.185732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.185877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.186024 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.186161 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.186297 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.186432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.186567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.186711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.186849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.186985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.187121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.187368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.187520 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.187749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.187891 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.188027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.188175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.188320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.188457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.188724 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.188884 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.189024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 18:15:09.189160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 18:15:09.189160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 18:15:09.192761 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.192934 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.193078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.193216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.193366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.193509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.201925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.202080 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.231112 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.231340 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.231487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.231640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.284404 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.285018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.336249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.336482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.336849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.337080 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.404389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.404570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.404644 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.404798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.404908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.405038 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.405162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.405246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.405436 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.405531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.405751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.406025 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.429150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.429390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.429493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.429614 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.429802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.429886 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.429949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.430010 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.430271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.430355 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.430419 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.430481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.430541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.430820 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.441718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.442013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.456244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.456374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.456459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.456526 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.456793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.456879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.456943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.457014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.457078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.457141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.457459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.457550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.457615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.457686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 18:15:09.475460 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.475667 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 18:15:09.475991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.476142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.476243 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 18:15:09.476310 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 18:15:09.476510 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.476652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.476732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.476906 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 18:15:09.476998 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.477064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.477259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.477340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.477576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.477833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.488099 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.488421 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.488688 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.488850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.488953 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.489180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.489402 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.489680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.497082 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.497853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.500367 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.500647 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.500925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.501151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.501317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.501416 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.501588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.501734 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.501876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.502100 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.502187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.502367 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.502775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.502909 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.502989 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.503163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.503415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.503559 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.503697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.503697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.503962 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.504233 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.530979 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.532649 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.532922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.536105 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.538283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.538569 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.538840 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.539127 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.539429 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.543991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.544280 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.544561 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.546816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.547097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.551742 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.551889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.552023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.552166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.552298 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.552573 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.552842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.553108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.553370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.553630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.556786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.557075 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.557357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.557636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.557905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.558174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.558441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.558706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.572706 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.572979 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.577512 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.577793 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.578067 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.581155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.581446 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.582586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.586766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.586974 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.587062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.590571 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.592018 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.594330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.594616 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.594883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.595197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.595474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.595622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.595775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.599437 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.599741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.600025 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.600309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.600582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.600859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.615331 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.615752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.616023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.616285 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.619793 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.620174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.620445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.620708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.624108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.625814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.626085 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.628863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.629374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.629672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.630458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.630748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.631019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.631104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.631404 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.634469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.634752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.635029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.635337 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.635613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.644455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.644731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.645002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.645285 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.646112 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.646402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.646669 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.649561 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.649838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.650115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.650405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.650685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.650972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.651280 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.664596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.667860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.669256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.671141 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.672568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.672736 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.674357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.674639 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.674918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.675882 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.677505 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.679156 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.679457 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.679747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.682316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.683931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.687118 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.687290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.687440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.687690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.687958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.688762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.692022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.692846 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.693128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.693401 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.697657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.697948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.698223 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.702584 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.702882 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.703164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.706198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=79826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.706479 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.706753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.707020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.707306 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.707588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.707874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.708691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.708955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.709216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.709477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.709750 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.710007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.710256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=79826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.711664 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=79825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 18:15:09.711947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.712225 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.712498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.712778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.713069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.713340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.713615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.713894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.714162 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.714428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.714703 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.714965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.715240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=79825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 18:15:09.748664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b3820469c0 pid: 79826 (default app) May 09 18:15:09.751962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542fcc7-f606-4a4d-9ea5-9597ddffb6e0 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:09.752254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542fcc7-f606-4a4d-9ea5-9597ddffb6e0 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:09.752792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542fcc7-f606-4a4d-9ea5-9597ddffb6e0 None None] PATH_INFO: `/v3` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:09.753990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b3820469c0 pid: 79825 (default app) May 09 18:15:09.757474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Mon May 9 18:15:09 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:17.690807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d926d9-ae5c-4a4e-8f5c-879b79f8c350 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:17.691393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d926d9-ae5c-4a4e-8f5c-879b79f8c350 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:17.691393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d926d9-ae5c-4a4e-8f5c-879b79f8c350 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:17.696398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:17.701656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256725f0-9e2d-4ec8-9c63-ae631ee1b6d2 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:17.702094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256725f0-9e2d-4ec8-9c63-ae631ee1b6d2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:17.702217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256725f0-9e2d-4ec8-9c63-ae631ee1b6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:17.735217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG oslo_db.sqlalchemy.engines [None req-256725f0-9e2d-4ec8-9c63-ae631ee1b6d2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=79826) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 18:15:17.919241 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG passlib.handlers.bcrypt [None req-256725f0-9e2d-4ec8-9c63-ae631ee1b6d2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=79826) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 18:15:17.919666 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG passlib.handlers.bcrypt [None req-256725f0-9e2d-4ec8-9c63-ae631ee1b6d2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=79826) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 18:15:17.976115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-256725f0-9e2d-4ec8-9c63-ae631ee1b6d2 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:17.977723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-256725f0-9e2d-4ec8-9c63-ae631ee1b6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:18.105479 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 09 18:15:18.282382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d926d9-ae5c-4a4e-8f5c-879b79f8c350 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:18.316108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG oslo_db.sqlalchemy.engines [None req-e7d926d9-ae5c-4a4e-8f5c-879b79f8c350 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=79825) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 18:15:18.443999 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d926d9-ae5c-4a4e-8f5c-879b79f8c350 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:18.448171 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cd9dd-ebb8-4735-be7f-20df2e35ac67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:18.448825 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cd9dd-ebb8-4735-be7f-20df2e35ac67 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:18.449108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cd9dd-ebb8-4735-be7f-20df2e35ac67 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:18.449379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cd9dd-ebb8-4735-be7f-20df2e35ac67 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:18.457365 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4cd9dd-ebb8-4735-be7f-20df2e35ac67 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:18.509383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ba4cd9dd-ebb8-4735-be7f-20df2e35ac67 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 18:15:18.509721 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4cd9dd-ebb8-4735-be7f-20df2e35ac67 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:18.519043 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Mon May 9 18:15:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 09 18:15:18.525005 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256725f0-9e2d-4ec8-9c63-ae631ee1b6d2 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:18.539213 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256725f0-9e2d-4ec8-9c63-ae631ee1b6d2 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:18.543588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79c090-d0df-494a-9ce3-61ff78fbb76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:18.544776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79c090-d0df-494a-9ce3-61ff78fbb76d admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:18.545132 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79c090-d0df-494a-9ce3-61ff78fbb76d admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:18.545475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79c090-d0df-494a-9ce3-61ff78fbb76d admin admin] PATH_INFO: `/v3/projects/969d4448da94407dbcabadb61787000d` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:18.547046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79c090-d0df-494a-9ce3-61ff78fbb76d admin admin] RBAC: Authorizing `identity:get_project(project_id=969d4448da94407dbcabadb61787000d)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:18.597684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8c79c090-d0df-494a-9ce3-61ff78fbb76d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 18:15:18.598025 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79c090-d0df-494a-9ce3-61ff78fbb76d admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:18.603041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Mon May 9 18:15:18 2022] GET /identity/v3/projects/969d4448da94407dbcabadb61787000d => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:15:20.796980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d45c45-a4d0-4e70-b877-5c4c5d7fbb74 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:20.797539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d45c45-a4d0-4e70-b877-5c4c5d7fbb74 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:20.797539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d45c45-a4d0-4e70-b877-5c4c5d7fbb74 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:20.801686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:20.807190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3638898-55cf-45a7-a034-7d3e4f0cfcb6 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:20.807614 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3638898-55cf-45a7-a034-7d3e4f0cfcb6 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:20.807765 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3638898-55cf-45a7-a034-7d3e4f0cfcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:20.827703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c3638898-55cf-45a7-a034-7d3e4f0cfcb6 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:20.829195 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c3638898-55cf-45a7-a034-7d3e4f0cfcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:20.837550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 09 18:15:20.858993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d45c45-a4d0-4e70-b877-5c4c5d7fbb74 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:20.870696 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d45c45-a4d0-4e70-b877-5c4c5d7fbb74 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:20.874824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d8076-40c5-4860-88f3-ebbc221c6a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:20.875466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d8076-40c5-4860-88f3-ebbc221c6a50 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:20.875750 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d8076-40c5-4860-88f3-ebbc221c6a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:20.876025 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d8076-40c5-4860-88f3-ebbc221c6a50 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:20.884235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d8076-40c5-4860-88f3-ebbc221c6a50 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:20.886002 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d8076-40c5-4860-88f3-ebbc221c6a50 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:20.894437 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Mon May 9 18:15:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 18:15:20.900170 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3638898-55cf-45a7-a034-7d3e4f0cfcb6 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:20.908867 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3638898-55cf-45a7-a034-7d3e4f0cfcb6 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:20.913210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc799593-d0ff-45a1-86f6-6044f93430ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:20.914070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc799593-d0ff-45a1-86f6-6044f93430ed admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:20.914354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc799593-d0ff-45a1-86f6-6044f93430ed admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:20.914630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc799593-d0ff-45a1-86f6-6044f93430ed admin admin] PATH_INFO: `/v3/users/a379c334e6cd446c80b18fda6ce9162e` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:20.916025 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc799593-d0ff-45a1-86f6-6044f93430ed admin admin] RBAC: Authorizing `identity:get_user(user_id=a379c334e6cd446c80b18fda6ce9162e)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:20.917807 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc799593-d0ff-45a1-86f6-6044f93430ed admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:20.919379 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Mon May 9 18:15:20 2022] GET /identity/v3/users/a379c334e6cd446c80b18fda6ce9162e => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:23.154819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ee5e7-efd0-47d4-b748-416312f75ba5 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:23.155378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ee5e7-efd0-47d4-b748-416312f75ba5 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:23.155378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ee5e7-efd0-47d4-b748-416312f75ba5 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:23.156750 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:23.161944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833313e-e546-4fb6-9735-74ea44125986 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:23.162371 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833313e-e546-4fb6-9735-74ea44125986 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:23.162596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833313e-e546-4fb6-9735-74ea44125986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:23.225734 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e91874-351b-420e-afb2-60d069b8a2ea None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:23.225919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e91874-351b-420e-afb2-60d069b8a2ea None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:23.226186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e91874-351b-420e-afb2-60d069b8a2ea None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:23.227587 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:15:23.233743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f30270-d1f9-4b10-b65a-6b5d2f4806ec None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:23.234030 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f30270-d1f9-4b10-b65a-6b5d2f4806ec None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:23.234311 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f30270-d1f9-4b10-b65a-6b5d2f4806ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:23.276933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG passlib.handlers.bcrypt [None req-97f30270-d1f9-4b10-b65a-6b5d2f4806ec None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=79825) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 18:15:23.277362 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG passlib.handlers.bcrypt [None req-97f30270-d1f9-4b10-b65a-6b5d2f4806ec None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=79825) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 18:15:23.287798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b833313e-e546-4fb6-9735-74ea44125986 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:23.289001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b833313e-e546-4fb6-9735-74ea44125986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:23.323737 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-97f30270-d1f9-4b10-b65a-6b5d2f4806ec None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:23.325290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-97f30270-d1f9-4b10-b65a-6b5d2f4806ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:23.368021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:23.391125 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833313e-e546-4fb6-9735-74ea44125986 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:23.423274 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:15:23.445625 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f30270-d1f9-4b10-b65a-6b5d2f4806ec None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:23.491527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833313e-e546-4fb6-9735-74ea44125986 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:23.522942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b0104-2a6f-4efa-ab59-fe39874ebeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:23.523751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b0104-2a6f-4efa-ab59-fe39874ebeca None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:23.524040 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b0104-2a6f-4efa-ab59-fe39874ebeca None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:23.524311 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b0104-2a6f-4efa-ab59-fe39874ebeca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:23.534719 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b0104-2a6f-4efa-ab59-fe39874ebeca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:23.538110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f30270-d1f9-4b10-b65a-6b5d2f4806ec None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-289b0104-2a6f-4efa-ab59-fe39874ebeca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:23.545741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:23.546883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:23.548107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:23.548107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:23.548107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:23.548107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:23.548107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:23.548107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:23.548107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:23.548107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:23.548107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:23.548107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:23.548107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:23.548107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:15:23.548107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:15:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:15:23.553077 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b0104-2a6f-4efa-ab59-fe39874ebeca None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:23.572953 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6adf2c-7398-452e-aeb4-c77dddd4b645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:23.573853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6adf2c-7398-452e-aeb4-c77dddd4b645 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:23.574135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6adf2c-7398-452e-aeb4-c77dddd4b645 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:23.574407 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6adf2c-7398-452e-aeb4-c77dddd4b645 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:23.637359 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6adf2c-7398-452e-aeb4-c77dddd4b645 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:23.638857 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6adf2c-7398-452e-aeb4-c77dddd4b645 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:23.645993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b0104-2a6f-4efa-ab59-fe39874ebeca None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:23.677576 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a79fbe-d7e0-44f4-849a-9f8a61855c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:23.678228 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a79fbe-d7e0-44f4-849a-9f8a61855c79 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:23.678511 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a79fbe-d7e0-44f4-849a-9f8a61855c79 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:23.678787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a79fbe-d7e0-44f4-849a-9f8a61855c79 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:23.679561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a79fbe-d7e0-44f4-849a-9f8a61855c79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:23.679877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a79fbe-d7e0-44f4-849a-9f8a61855c79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:23.681240 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a79fbe-d7e0-44f4-849a-9f8a61855c79 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:23.693139 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:15:23.698365 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a79fbe-d7e0-44f4-849a-9f8a61855c79 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:23.729318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:15:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 18:15:23.734372 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6adf2c-7398-452e-aeb4-c77dddd4b645 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:23.790463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a79fbe-d7e0-44f4-849a-9f8a61855c79 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:23.823925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddac94-25f2-49f3-badd-2ddb50086397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:23.824790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddac94-25f2-49f3-badd-2ddb50086397 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:23.825081 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddac94-25f2-49f3-badd-2ddb50086397 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:23.825357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddac94-25f2-49f3-badd-2ddb50086397 None admin] PATH_INFO: `/v3/users/a379c334e6cd446c80b18fda6ce9162e` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:23.826554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbddac94-25f2-49f3-badd-2ddb50086397 None admin] RBAC: Authorizing `identity:get_user(user_id=a379c334e6cd446c80b18fda6ce9162e)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:23.827757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6adf2c-7398-452e-aeb4-c77dddd4b645 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:23.828021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbddac94-25f2-49f3-badd-2ddb50086397 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:23.829774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:23 2022] GET /identity/v3/users/a379c334e6cd446c80b18fda6ce9162e => generated 247 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:15:23.834903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddac94-25f2-49f3-badd-2ddb50086397 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:23.859341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed64b4-5282-44f1-bf19-68130e7097c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:23.860134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed64b4-5282-44f1-bf19-68130e7097c4 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:23.860415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed64b4-5282-44f1-bf19-68130e7097c4 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:23.860683 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed64b4-5282-44f1-bf19-68130e7097c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:23.932129 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddac94-25f2-49f3-badd-2ddb50086397 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:23.966712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59f6fb-e2a6-450c-a85e-6403db2b3f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:23.967562 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59f6fb-e2a6-450c-a85e-6403db2b3f5c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:23.967851 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59f6fb-e2a6-450c-a85e-6403db2b3f5c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:23.968129 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59f6fb-e2a6-450c-a85e-6403db2b3f5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:23.969394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59f6fb-e2a6-450c-a85e-6403db2b3f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:23.970835 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59f6fb-e2a6-450c-a85e-6403db2b3f5c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:23.972654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 10/18] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:15:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:15:23.978553 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59f6fb-e2a6-450c-a85e-6403db2b3f5c None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:23.986468 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed64b4-5282-44f1-bf19-68130e7097c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:23.987958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed64b4-5282-44f1-bf19-68130e7097c4 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-58ed64b4-5282-44f1-bf19-68130e7097c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:15:24.008879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:24.009732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:24.011007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:15:24.011007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:15:24.011007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:15:24.011007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:24.011007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 9/19] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:15:24.016420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed64b4-5282-44f1-bf19-68130e7097c4 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:24.073056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59f6fb-e2a6-450c-a85e-6403db2b3f5c None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:24.104537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd9ca6-992e-4f92-a1a0-1ab4030f5457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:24.105187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd9ca6-992e-4f92-a1a0-1ab4030f5457 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:24.105472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd9ca6-992e-4f92-a1a0-1ab4030f5457 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:24.105753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd9ca6-992e-4f92-a1a0-1ab4030f5457 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:24.106578 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd9ca6-992e-4f92-a1a0-1ab4030f5457 None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.domain.id=default, user.id=a379c334e6cd446c80b18fda6ce9162e) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:24.106890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd9ca6-992e-4f92-a1a0-1ab4030f5457 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:24.108346 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd9ca6-992e-4f92-a1a0-1ab4030f5457 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:24.108717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed64b4-5282-44f1-bf19-68130e7097c4 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:24.115328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 11/20] 10.222.0.25 () {56 vars in 1322 bytes} [Mon May 9 18:15:23 2022] GET /identity/v3/role_assignments?user.id=a379c334e6cd446c80b18fda6ce9162e&scope.domain.id=default&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 233 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:15:24.143259 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1948ce-59f9-4df4-8d9a-352dd1b49c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:24.143953 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1948ce-59f9-4df4-8d9a-352dd1b49c4c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:24.144232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1948ce-59f9-4df4-8d9a-352dd1b49c4c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:24.144505 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1948ce-59f9-4df4-8d9a-352dd1b49c4c None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:24.145315 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1948ce-59f9-4df4-8d9a-352dd1b49c4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:24.145622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1948ce-59f9-4df4-8d9a-352dd1b49c4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:24.147050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1948ce-59f9-4df4-8d9a-352dd1b49c4c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:24.168687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 10/21] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:15:26.840229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9b32e-1b1b-4706-a25a-5dc401d45a01 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:26.840728 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9b32e-1b1b-4706-a25a-5dc401d45a01 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:26.840800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9b32e-1b1b-4706-a25a-5dc401d45a01 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:26.842230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 12/22] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:26.847683 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d162d-fab8-4d3a-986c-227c407d73ab None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:26.848097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d162d-fab8-4d3a-986c-227c407d73ab None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:26.848803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d162d-fab8-4d3a-986c-227c407d73ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:26.870280 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-0c4d162d-fab8-4d3a-986c-227c407d73ab None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:26.871487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-0c4d162d-fab8-4d3a-986c-227c407d73ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:26.947429 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0f30c-12c9-4c07-8730-4a4eb24ff563 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:26.948071 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0f30c-12c9-4c07-8730-4a4eb24ff563 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:26.948554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0f30c-12c9-4c07-8730-4a4eb24ff563 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:26.950373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 13/23] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:26.955900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4549cd5-7a08-433f-a3d5-45e57c8eeb8b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:26.956423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4549cd5-7a08-433f-a3d5-45e57c8eeb8b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:26.956919 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4549cd5-7a08-433f-a3d5-45e57c8eeb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:26.957779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 11/24] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:26.984548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d162d-fab8-4d3a-986c-227c407d73ab None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:26.988750 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-f4549cd5-7a08-433f-a3d5-45e57c8eeb8b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:26.990776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f4549cd5-7a08-433f-a3d5-45e57c8eeb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:27.112389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 14/25] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:15:27.136236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d162d-fab8-4d3a-986c-227c407d73ab None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:27.157822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4549cd5-7a08-433f-a3d5-45e57c8eeb8b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:27.183669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d26afc-47dc-4e7a-8d44-78a56fd3a637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:27.184504 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26afc-47dc-4e7a-8d44-78a56fd3a637 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:27.184787 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26afc-47dc-4e7a-8d44-78a56fd3a637 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:27.185130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26afc-47dc-4e7a-8d44-78a56fd3a637 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:27.220929 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d26afc-47dc-4e7a-8d44-78a56fd3a637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-08d26afc-47dc-4e7a-8d44-78a56fd3a637 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:27.233838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:27.234741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:27.235521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:27.235521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:27.235521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:27.235521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:27.235521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:27.235521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:27.235521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:27.235521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:27.235521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:27.235521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:27.235521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:27.235521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:27.235521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 12/26] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:15:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:15:27.242280 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d26afc-47dc-4e7a-8d44-78a56fd3a637 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:27.358880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4549cd5-7a08-433f-a3d5-45e57c8eeb8b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:27.404147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d26afc-47dc-4e7a-8d44-78a56fd3a637 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:27.415110 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a62a62-4132-4470-9317-f35074559167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:27.416112 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a62a62-4132-4470-9317-f35074559167 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:27.416598 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a62a62-4132-4470-9317-f35074559167 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:27.417092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a62a62-4132-4470-9317-f35074559167 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:27.418052 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a62a62-4132-4470-9317-f35074559167 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:27.419688 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a62a62-4132-4470-9317-f35074559167 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:27.477342 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e609f43-ab48-4414-aa10-abdd82dce557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:27.478043 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e609f43-ab48-4414-aa10-abdd82dce557 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:27.478326 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e609f43-ab48-4414-aa10-abdd82dce557 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:27.478601 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e609f43-ab48-4414-aa10-abdd82dce557 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:27.479396 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e609f43-ab48-4414-aa10-abdd82dce557 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:27.479701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e609f43-ab48-4414-aa10-abdd82dce557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:27.481140 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e609f43-ab48-4414-aa10-abdd82dce557 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:27.503822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 13/27] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:27.511197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf74ea5-428d-49e4-8eca-68d021bdbb31 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:27.511197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf74ea5-428d-49e4-8eca-68d021bdbb31 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:27.511197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf74ea5-428d-49e4-8eca-68d021bdbb31 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:27.511197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 14/28] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:27.512560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf74ea5-428d-49e4-8eca-68d021bdbb31 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:27.570585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 15/29] 10.222.0.25 () {60 vars in 1131 bytes} [Mon May 9 18:15:27 2022] POST /identity/v3/roles => generated 219 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:15:27.580352 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4424221-56d3-4ebc-930c-bcf76ba3e0ae None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:27.581468 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4424221-56d3-4ebc-930c-bcf76ba3e0ae None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:27.581970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4424221-56d3-4ebc-930c-bcf76ba3e0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:27.608434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-d4424221-56d3-4ebc-930c-bcf76ba3e0ae None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:27.610618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-d4424221-56d3-4ebc-930c-bcf76ba3e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:27.689427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 16/30] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:27.714192 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4424221-56d3-4ebc-930c-bcf76ba3e0ae None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:27.747992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf74ea5-428d-49e4-8eca-68d021bdbb31 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:27.791019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e1432-f75d-437f-b537-292491aad74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:27.791943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e1432-f75d-437f-b537-292491aad74d None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:27.792224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e1432-f75d-437f-b537-292491aad74d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:27.792495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e1432-f75d-437f-b537-292491aad74d None admin] PATH_INFO: `/v3/users/a379c334e6cd446c80b18fda6ce9162e` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:27.794863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e1432-f75d-437f-b537-292491aad74d None admin] RBAC: Authorizing `identity:get_user(user_id=a379c334e6cd446c80b18fda6ce9162e)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:27.796372 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e1432-f75d-437f-b537-292491aad74d None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:27.799427 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 15/31] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:27 2022] GET /identity/v3/users/a379c334e6cd446c80b18fda6ce9162e => generated 247 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:27.805280 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0916da-5a92-4968-9c36-22f5235a3aa1 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:27.807301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0916da-5a92-4968-9c36-22f5235a3aa1 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:27.807746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0916da-5a92-4968-9c36-22f5235a3aa1 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:27.811381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 16/32] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:27.812693 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0916da-5a92-4968-9c36-22f5235a3aa1 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:27.877102 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4424221-56d3-4ebc-930c-bcf76ba3e0ae None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:27.914779 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e2963-25c8-4875-a170-1f7c65435631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:27.915927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e2963-25c8-4875-a170-1f7c65435631 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:27.916348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e2963-25c8-4875-a170-1f7c65435631 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:27.916786 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e2963-25c8-4875-a170-1f7c65435631 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:27.927866 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e2963-25c8-4875-a170-1f7c65435631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:27.929756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e2963-25c8-4875-a170-1f7c65435631 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:27.943283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0916da-5a92-4968-9c36-22f5235a3aa1 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-881e2963-25c8-4875-a170-1f7c65435631 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:15:27.952899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:27.953859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:27.954668 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:15:27.954668 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:15:27.954668 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:15:27.954668 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:15:27.955384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 17/33] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:27.956625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f013b-5e7a-4689-9834-62e3c6561b71 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:27.956789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f013b-5e7a-4689-9834-62e3c6561b71 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:27.957038 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f013b-5e7a-4689-9834-62e3c6561b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:27.974027 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-fb6f013b-5e7a-4689-9834-62e3c6561b71 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:27.975231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-fb6f013b-5e7a-4689-9834-62e3c6561b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:27.988046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43757011-9a5b-40f6-a6ad-f751e980f0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:27.989387 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43757011-9a5b-40f6-a6ad-f751e980f0c7 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:27.989672 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43757011-9a5b-40f6-a6ad-f751e980f0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:27.989948 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43757011-9a5b-40f6-a6ad-f751e980f0c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:27.991143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43757011-9a5b-40f6-a6ad-f751e980f0c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:27.992617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43757011-9a5b-40f6-a6ad-f751e980f0c7 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:27.998702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 17/34] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:15:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:27.998702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5eac1f-519c-4f17-ad3d-4f5b3540de6b None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:27.998702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5eac1f-519c-4f17-ad3d-4f5b3540de6b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:27.998702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5eac1f-519c-4f17-ad3d-4f5b3540de6b None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:27.998702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 18/35] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:27.998702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5eac1f-519c-4f17-ad3d-4f5b3540de6b None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:28.048095 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 18/36] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:28.049235 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f013b-5e7a-4689-9834-62e3c6561b71 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:28.112380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5eac1f-519c-4f17-ad3d-4f5b3540de6b None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:28.144325 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f12c94-d3ab-4ae5-a922-ba06bacc91e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:28.144992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f12c94-d3ab-4ae5-a922-ba06bacc91e9 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:28.145803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f12c94-d3ab-4ae5-a922-ba06bacc91e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:28.146073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f12c94-d3ab-4ae5-a922-ba06bacc91e9 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:28.146787 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f12c94-d3ab-4ae5-a922-ba06bacc91e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:28.147089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f12c94-d3ab-4ae5-a922-ba06bacc91e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:28.148428 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f013b-5e7a-4689-9834-62e3c6561b71 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:28.148543 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f12c94-d3ab-4ae5-a922-ba06bacc91e9 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:28.162064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 19/37] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:28.163113 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987553b-c5fd-45c2-afba-524fe4000847 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:28.163419 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987553b-c5fd-45c2-afba-524fe4000847 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:28.163694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987553b-c5fd-45c2-afba-524fe4000847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:28.179226 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-8987553b-c5fd-45c2-afba-524fe4000847 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:28.180401 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-8987553b-c5fd-45c2-afba-524fe4000847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:28.185866 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc737ce3-3576-4140-93b5-dd91baf1e34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:28.187002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc737ce3-3576-4140-93b5-dd91baf1e34a None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:28.187933 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc737ce3-3576-4140-93b5-dd91baf1e34a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:28.188413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc737ce3-3576-4140-93b5-dd91baf1e34a None admin] PATH_INFO: `/v3/domains/default/users/a379c334e6cd446c80b18fda6ce9162e/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:28.191238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc737ce3-3576-4140-93b5-dd91baf1e34a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a379c334e6cd446c80b18fda6ce9162e, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:28.193281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc737ce3-3576-4140-93b5-dd91baf1e34a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:28.210756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 19/38] 10.222.0.25 () {58 vars in 1361 bytes} [Mon May 9 18:15:28 2022] PUT /identity/v3/domains/default/users/a379c334e6cd446c80b18fda6ce9162e/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:15:28.215434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc737ce3-3576-4140-93b5-dd91baf1e34a None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:28.275677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 20/39] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:28.276882 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987553b-c5fd-45c2-afba-524fe4000847 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:28.329274 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc737ce3-3576-4140-93b5-dd91baf1e34a None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:28.364528 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f106a2-af6d-4c0f-99b2-4a5df40c72b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:28.365214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f106a2-af6d-4c0f-99b2-4a5df40c72b5 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:28.365505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f106a2-af6d-4c0f-99b2-4a5df40c72b5 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:28.365788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f106a2-af6d-4c0f-99b2-4a5df40c72b5 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:28.366548 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f106a2-af6d-4c0f-99b2-4a5df40c72b5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:28.368009 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f106a2-af6d-4c0f-99b2-4a5df40c72b5 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:28.386969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 20/40] 10.222.0.25 () {60 vars in 1131 bytes} [Mon May 9 18:15:28 2022] POST /identity/v3/roles => generated 223 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:15:28.391343 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f106a2-af6d-4c0f-99b2-4a5df40c72b5 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:28.440706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987553b-c5fd-45c2-afba-524fe4000847 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:28.494814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15af389-154f-4c5d-a21d-b830c743f229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:28.495600 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15af389-154f-4c5d-a21d-b830c743f229 None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:28.495882 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15af389-154f-4c5d-a21d-b830c743f229 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:28.496154 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15af389-154f-4c5d-a21d-b830c743f229 None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:28.496883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15af389-154f-4c5d-a21d-b830c743f229 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:28.498366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15af389-154f-4c5d-a21d-b830c743f229 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:28.534345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f106a2-af6d-4c0f-99b2-4a5df40c72b5 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:28.571968 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334eda8-4cc3-4bf7-a56e-eff444f1c4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:28.572689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334eda8-4cc3-4bf7-a56e-eff444f1c4c9 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:28.572979 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334eda8-4cc3-4bf7-a56e-eff444f1c4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:28.573304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334eda8-4cc3-4bf7-a56e-eff444f1c4c9 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:28.574117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334eda8-4cc3-4bf7-a56e-eff444f1c4c9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:28.575713 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334eda8-4cc3-4bf7-a56e-eff444f1c4c9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:28.593495 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 21/41] 10.222.0.25 () {60 vars in 1131 bytes} [Mon May 9 18:15:28 2022] POST /identity/v3/roles => generated 225 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:15:28.594886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa4cc1-a3ac-40f4-903d-3a2d22aa7649 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:28.595199 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa4cc1-a3ac-40f4-903d-3a2d22aa7649 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:28.603997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa4cc1-a3ac-40f4-903d-3a2d22aa7649 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:28.605772 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 22/42] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:28 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:28.617264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4d344-394c-4435-94dc-f8197539d9c5 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:28.617549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4d344-394c-4435-94dc-f8197539d9c5 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:28.617921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4d344-394c-4435-94dc-f8197539d9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:28.638916 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-fcd4d344-394c-4435-94dc-f8197539d9c5 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:28.640229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-fcd4d344-394c-4435-94dc-f8197539d9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:28.661733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 21/43] 10.222.0.25 () {60 vars in 1141 bytes} [Mon May 9 18:15:28 2022] POST /identity/v3/projects => generated 301 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:15:28.730591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 23/44] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:28.760692 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15af389-154f-4c5d-a21d-b830c743f229 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:28.889428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15af389-154f-4c5d-a21d-b830c743f229 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:28.922877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af046f-f5bb-409d-9a6f-cc907afd6ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:28.923711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af046f-f5bb-409d-9a6f-cc907afd6ed7 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:28.923998 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af046f-f5bb-409d-9a6f-cc907afd6ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:28.924275 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af046f-f5bb-409d-9a6f-cc907afd6ed7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:28.925527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af046f-f5bb-409d-9a6f-cc907afd6ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:28.927642 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af046f-f5bb-409d-9a6f-cc907afd6ed7 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:28.929264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 22/45] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:15:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:28.934868 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4d344-394c-4435-94dc-f8197539d9c5 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:29.029629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4d344-394c-4435-94dc-f8197539d9c5 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:29.062170 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9646e5b-d204-4487-a93f-c373e6e69952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:29.062830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9646e5b-d204-4487-a93f-c373e6e69952 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:29.063110 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9646e5b-d204-4487-a93f-c373e6e69952 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:29.063411 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9646e5b-d204-4487-a93f-c373e6e69952 None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:29.064116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9646e5b-d204-4487-a93f-c373e6e69952 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:29.065531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9646e5b-d204-4487-a93f-c373e6e69952 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:29.091052 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 24/46] 10.222.0.25 () {60 vars in 1141 bytes} [Mon May 9 18:15:28 2022] POST /identity/v3/projects => generated 312 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:15:30.515875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535eb81-3438-4971-a789-48c4a47f2d00 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:30.516392 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535eb81-3438-4971-a789-48c4a47f2d00 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:30.516502 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535eb81-3438-4971-a789-48c4a47f2d00 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:30.517803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 23/47] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:30.523129 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92bfed-5837-4dcf-9c5f-095a4271a863 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:30.523546 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92bfed-5837-4dcf-9c5f-095a4271a863 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:30.523717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92bfed-5837-4dcf-9c5f-095a4271a863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:30.540201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-3c92bfed-5837-4dcf-9c5f-095a4271a863 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:30.541362 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3c92bfed-5837-4dcf-9c5f-095a4271a863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:30.602755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 25/48] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:30.626406 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535eb81-3438-4971-a789-48c4a47f2d00 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:30.725161 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535eb81-3438-4971-a789-48c4a47f2d00 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:30.766298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf435c5-3921-4d0e-8f5d-217ce871e749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:30.767031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf435c5-3921-4d0e-8f5d-217ce871e749 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:30.767349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf435c5-3921-4d0e-8f5d-217ce871e749 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:30.767632 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf435c5-3921-4d0e-8f5d-217ce871e749 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:30.777394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf435c5-3921-4d0e-8f5d-217ce871e749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-fdf435c5-3921-4d0e-8f5d-217ce871e749 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:30.785033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:30.787236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:30.788965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:30.788965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:30.788965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:30.788965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:30.788965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:30.788965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:30.788965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:30.788965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:30.788965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:30.788965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:30.788965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:30.788965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:30.788965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 24/49] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:15:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:30.792476 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92bfed-5837-4dcf-9c5f-095a4271a863 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:30.885010 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92bfed-5837-4dcf-9c5f-095a4271a863 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:30.916133 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c99fc5-7b38-49c1-8f71-8f6b35544a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:30.917143 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c99fc5-7b38-49c1-8f71-8f6b35544a03 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:30.917363 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c99fc5-7b38-49c1-8f71-8f6b35544a03 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:30.917636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c99fc5-7b38-49c1-8f71-8f6b35544a03 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:30.918377 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c99fc5-7b38-49c1-8f71-8f6b35544a03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:30.918681 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c99fc5-7b38-49c1-8f71-8f6b35544a03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:30.920078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c99fc5-7b38-49c1-8f71-8f6b35544a03 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:30.928416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 26/50] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:30.933976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf435c5-3921-4d0e-8f5d-217ce871e749 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:31.029670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf435c5-3921-4d0e-8f5d-217ce871e749 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:31.062945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42fd37-f9e9-43ed-8c17-d391c8990851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:31.063841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42fd37-f9e9-43ed-8c17-d391c8990851 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:31.064126 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42fd37-f9e9-43ed-8c17-d391c8990851 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:31.064402 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42fd37-f9e9-43ed-8c17-d391c8990851 None admin] PATH_INFO: `/v3/users/a379c334e6cd446c80b18fda6ce9162e` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:31.065662 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42fd37-f9e9-43ed-8c17-d391c8990851 None admin] RBAC: Authorizing `identity:get_user(user_id=a379c334e6cd446c80b18fda6ce9162e)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:31.067121 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42fd37-f9e9-43ed-8c17-d391c8990851 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:31.068756 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 25/51] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:30 2022] GET /identity/v3/users/a379c334e6cd446c80b18fda6ce9162e => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:31.074059 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c99fc5-7b38-49c1-8f71-8f6b35544a03 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:31.171601 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c99fc5-7b38-49c1-8f71-8f6b35544a03 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:31.201787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e928b2a-ca57-4d6a-96e8-73203acc160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:31.202582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e928b2a-ca57-4d6a-96e8-73203acc160c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:31.202862 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e928b2a-ca57-4d6a-96e8-73203acc160c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:31.203136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e928b2a-ca57-4d6a-96e8-73203acc160c None admin] PATH_INFO: `/v3/domains/default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:31.204308 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e928b2a-ca57-4d6a-96e8-73203acc160c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:31.205735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e928b2a-ca57-4d6a-96e8-73203acc160c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:31.207332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 27/52] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:15:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:31.213534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42fd37-f9e9-43ed-8c17-d391c8990851 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:31.307827 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5e71b-eaec-4449-a545-a70edca517a8 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:31.308053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5e71b-eaec-4449-a545-a70edca517a8 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:31.308336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5e71b-eaec-4449-a545-a70edca517a8 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:31.309693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 28/53] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:31.313001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42fd37-f9e9-43ed-8c17-d391c8990851 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:31.314672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b13a4-0efd-4a67-9626-52699b703374 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:31.314966 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b13a4-0efd-4a67-9626-52699b703374 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:31.315279 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b13a4-0efd-4a67-9626-52699b703374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:31.332089 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-e61b13a4-0efd-4a67-9626-52699b703374 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:31.333865 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e61b13a4-0efd-4a67-9626-52699b703374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:31.349054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93281a-3ebd-4580-bed3-d4d4c3f682f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:31.349723 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93281a-3ebd-4580-bed3-d4d4c3f682f9 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:31.350011 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93281a-3ebd-4580-bed3-d4d4c3f682f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:31.350290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93281a-3ebd-4580-bed3-d4d4c3f682f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:31.351128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93281a-3ebd-4580-bed3-d4d4c3f682f9 None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.domain.id=default, user.id=a379c334e6cd446c80b18fda6ce9162e) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:31.351470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93281a-3ebd-4580-bed3-d4d4c3f682f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:31.352933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93281a-3ebd-4580-bed3-d4d4c3f682f9 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:31.361399 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 26/54] 10.222.0.25 () {56 vars in 1322 bytes} [Mon May 9 18:15:31 2022] GET /identity/v3/role_assignments?user.id=a379c334e6cd446c80b18fda6ce9162e&scope.domain.id=default&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:31.401297 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 29/55] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:15:31.424179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93281a-3ebd-4580-bed3-d4d4c3f682f9 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:31.524720 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93281a-3ebd-4580-bed3-d4d4c3f682f9 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:31.557133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ead21e-c9d1-4cb2-adb5-13c99f9b7323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:31.557983 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ead21e-c9d1-4cb2-adb5-13c99f9b7323 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:31.558218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ead21e-c9d1-4cb2-adb5-13c99f9b7323 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:31.558497 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ead21e-c9d1-4cb2-adb5-13c99f9b7323 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:31.559709 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ead21e-c9d1-4cb2-adb5-13c99f9b7323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:31.561159 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ead21e-c9d1-4cb2-adb5-13c99f9b7323 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:31.562767 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 27/56] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:15:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:31.568813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b13a4-0efd-4a67-9626-52699b703374 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:31.663348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b13a4-0efd-4a67-9626-52699b703374 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:31.694652 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fdcb6-27ce-4918-a813-8a55835b7b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:31.695334 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fdcb6-27ce-4918-a813-8a55835b7b3e None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:31.695622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fdcb6-27ce-4918-a813-8a55835b7b3e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:31.695898 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fdcb6-27ce-4918-a813-8a55835b7b3e None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:31.696603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fdcb6-27ce-4918-a813-8a55835b7b3e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:31.698018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fdcb6-27ce-4918-a813-8a55835b7b3e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:31.721149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 30/57] 10.222.0.25 () {60 vars in 1140 bytes} [Mon May 9 18:15:31 2022] POST /identity/v3/projects => generated 298 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:15:33.933771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e8503-019a-43c8-aa45-7e5d9fe5d800 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:33.934289 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e8503-019a-43c8-aa45-7e5d9fe5d800 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:33.934370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e8503-019a-43c8-aa45-7e5d9fe5d800 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:33.935725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 28/58] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:33.941134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c1320-35c6-4a0f-b5d4-8a022818fbe1 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:33.941543 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c1320-35c6-4a0f-b5d4-8a022818fbe1 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:33.941692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c1320-35c6-4a0f-b5d4-8a022818fbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:33.958290 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-561c1320-35c6-4a0f-b5d4-8a022818fbe1 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:33.959584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-561c1320-35c6-4a0f-b5d4-8a022818fbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:34.023533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 31/59] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:34.047546 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e8503-019a-43c8-aa45-7e5d9fe5d800 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:34.142622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e8503-019a-43c8-aa45-7e5d9fe5d800 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:34.173373 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8dfd4-ba9b-49a8-a169-46b771bc059b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:34.174157 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8dfd4-ba9b-49a8-a169-46b771bc059b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:34.174445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8dfd4-ba9b-49a8-a169-46b771bc059b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:34.174726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8dfd4-ba9b-49a8-a169-46b771bc059b None admin] PATH_INFO: `/v3/domains/default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:34.175901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8dfd4-ba9b-49a8-a169-46b771bc059b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:34.177851 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8dfd4-ba9b-49a8-a169-46b771bc059b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:34.179473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 29/60] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:15:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:34.184766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c1320-35c6-4a0f-b5d4-8a022818fbe1 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:34.275610 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c1320-35c6-4a0f-b5d4-8a022818fbe1 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:34.306566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144c64f-c69b-4954-a8fd-59cace01efb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:34.307242 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144c64f-c69b-4954-a8fd-59cace01efb3 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:34.307526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144c64f-c69b-4954-a8fd-59cace01efb3 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:34.307796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144c64f-c69b-4954-a8fd-59cace01efb3 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:34.308483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144c64f-c69b-4954-a8fd-59cace01efb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:34.310240 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144c64f-c69b-4954-a8fd-59cace01efb3 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:34.335820 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 32/61] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:15:34 2022] POST /identity/v3/users => generated 275 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:15:37.523941 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340d75c-4c97-4626-aa82-33385fbd4e29 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:37.524861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340d75c-4c97-4626-aa82-33385fbd4e29 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:37.525311 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340d75c-4c97-4626-aa82-33385fbd4e29 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:37.526845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 30/62] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:37.532140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de1ebe-a0dc-4fc3-95ce-c1f029216d53 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:37.532938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de1ebe-a0dc-4fc3-95ce-c1f029216d53 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:37.533397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de1ebe-a0dc-4fc3-95ce-c1f029216d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:37.556190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-08de1ebe-a0dc-4fc3-95ce-c1f029216d53 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:37.557689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-08de1ebe-a0dc-4fc3-95ce-c1f029216d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:37.645244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 33/63] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:15:37.675271 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340d75c-4c97-4626-aa82-33385fbd4e29 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:37.812908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340d75c-4c97-4626-aa82-33385fbd4e29 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:37.858688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd968600-5235-4f23-91ea-36da4d9fb595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:37.859858 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd968600-5235-4f23-91ea-36da4d9fb595 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:37.860319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd968600-5235-4f23-91ea-36da4d9fb595 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:37.860769 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd968600-5235-4f23-91ea-36da4d9fb595 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:37.876237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd968600-5235-4f23-91ea-36da4d9fb595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-cd968600-5235-4f23-91ea-36da4d9fb595 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:37.884490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:37.885336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:37.886088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:37.886088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:37.886088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:37.886088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:37.886088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:37.886088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:37.886088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:37.886088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:37.886088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:37.886088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:37.886088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:15:37.886088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:37.888045 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 31/64] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:37.893714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de1ebe-a0dc-4fc3-95ce-c1f029216d53 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:37.990570 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc00b3-6e6f-48aa-a2b1-f06f47819822 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:37.991079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc00b3-6e6f-48aa-a2b1-f06f47819822 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:37.991395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc00b3-6e6f-48aa-a2b1-f06f47819822 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:37.992956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 32/65] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:15:37.999770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c71d1-1fbf-479b-976a-296015049cf5 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.000320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c71d1-1fbf-479b-976a-296015049cf5 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.000817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c71d1-1fbf-479b-976a-296015049cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.029118 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de1ebe-a0dc-4fc3-95ce-c1f029216d53 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:38.030992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-099c71d1-1fbf-479b-976a-296015049cf5 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:38.033275 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-099c71d1-1fbf-479b-976a-296015049cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:38.072735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f56d2-7bfc-416d-be57-417af79265bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:38.073775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f56d2-7bfc-416d-be57-417af79265bb None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.074449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f56d2-7bfc-416d-be57-417af79265bb None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.075889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f56d2-7bfc-416d-be57-417af79265bb None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.076904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f56d2-7bfc-416d-be57-417af79265bb None admin] RBAC: Adding query filter params (name=member) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:38.077392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f56d2-7bfc-416d-be57-417af79265bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:38.079011 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f56d2-7bfc-416d-be57-417af79265bb None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:38.092200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 34/66] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:15:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:38.094125 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e419e27-7ca6-4acd-97af-11cfd80e9b0e None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.094585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e419e27-7ca6-4acd-97af-11cfd80e9b0e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.095023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e419e27-7ca6-4acd-97af-11cfd80e9b0e None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.101693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 35/67] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:38.103310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e419e27-7ca6-4acd-97af-11cfd80e9b0e None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:38.116820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 33/68] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:38.123205 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8245c-5e54-485f-a579-77d95694803c None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.123736 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8245c-5e54-485f-a579-77d95694803c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.124157 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8245c-5e54-485f-a579-77d95694803c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.160649 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-10f8245c-5e54-485f-a579-77d95694803c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:38.162110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-10f8245c-5e54-485f-a579-77d95694803c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:38.228592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e419e27-7ca6-4acd-97af-11cfd80e9b0e None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:38.229765 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 34/69] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:38.230830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8245c-5e54-485f-a579-77d95694803c None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:38.282386 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e26793-d4d2-4cc3-8c1f-08c5b8d40f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:38.283300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e26793-d4d2-4cc3-8c1f-08c5b8d40f1c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.283586 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e26793-d4d2-4cc3-8c1f-08c5b8d40f1c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.283866 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e26793-d4d2-4cc3-8c1f-08c5b8d40f1c None admin] PATH_INFO: `/v3/users/416a6d6800844d94a22ac939f6f6ba2a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.298317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e26793-d4d2-4cc3-8c1f-08c5b8d40f1c None admin] RBAC: Authorizing `identity:get_user(user_id=416a6d6800844d94a22ac939f6f6ba2a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:38.299809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e26793-d4d2-4cc3-8c1f-08c5b8d40f1c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:38.301630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 36/70] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:38 2022] GET /identity/v3/users/416a6d6800844d94a22ac939f6f6ba2a => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:38.306780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e26793-d4d2-4cc3-8c1f-08c5b8d40f1c None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:38.347100 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8245c-5e54-485f-a579-77d95694803c None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:38.378355 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ad3fe-df1f-41ac-9ed3-3d5145e89bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:38.379667 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ad3fe-df1f-41ac-9ed3-3d5145e89bad None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.379965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ad3fe-df1f-41ac-9ed3-3d5145e89bad None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.380238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ad3fe-df1f-41ac-9ed3-3d5145e89bad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.391834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ad3fe-df1f-41ac-9ed3-3d5145e89bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-a21ad3fe-df1f-41ac-9ed3-3d5145e89bad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:38.400589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:38.402444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:38.404466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:38.404466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:38.404466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:38.404466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:38.404466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:38.404466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:38.404466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:38.404466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:38.404466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:38.404466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:38.404466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:15:38.404466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:38.404466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 35/71] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:15:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:38.404466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02d6c-b86c-4f59-af12-b09202ef520e None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.405795 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e26793-d4d2-4cc3-8c1f-08c5b8d40f1c None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:38.405922 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02d6c-b86c-4f59-af12-b09202ef520e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.405922 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae02d6c-b86c-4f59-af12-b09202ef520e None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.409411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 36/72] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:38.409411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae02d6c-b86c-4f59-af12-b09202ef520e None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:38.446554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6208ec-fafd-43e2-bbe1-8c4c27ea2993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:38.447396 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6208ec-fafd-43e2-bbe1-8c4c27ea2993 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.447689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6208ec-fafd-43e2-bbe1-8c4c27ea2993 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.447964 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6208ec-fafd-43e2-bbe1-8c4c27ea2993 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.458316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6208ec-fafd-43e2-bbe1-8c4c27ea2993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-6c6208ec-fafd-43e2-bbe1-8c4c27ea2993 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:38.468690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:38.469558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:38.470300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:38.470300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:38.470300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:38.470300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:38.470300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:38.470300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:38.470300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:38.470300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:38.470300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:38.470300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:38.470300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:38.470300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:15:38.470300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 37/73] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:15:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:38.471351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90974b51-c6c7-4a82-add2-cd3d1a71e57e None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.471604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90974b51-c6c7-4a82-add2-cd3d1a71e57e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.471870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90974b51-c6c7-4a82-add2-cd3d1a71e57e None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.473153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 38/74] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:38.474198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2803caf-effe-4d56-a59c-a1399188370b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.474497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2803caf-effe-4d56-a59c-a1399188370b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.474938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2803caf-effe-4d56-a59c-a1399188370b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.491286 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c2803caf-effe-4d56-a59c-a1399188370b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:38.492748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c2803caf-effe-4d56-a59c-a1399188370b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:38.515856 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae02d6c-b86c-4f59-af12-b09202ef520e None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:38.548324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9ea43-ad0e-4c3a-a665-235a8bff906c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:38.548977 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9ea43-ad0e-4c3a-a665-235a8bff906c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.549248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9ea43-ad0e-4c3a-a665-235a8bff906c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.549519 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9ea43-ad0e-4c3a-a665-235a8bff906c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.567859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 39/75] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:38.567859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2803caf-effe-4d56-a59c-a1399188370b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:38.616197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9ea43-ad0e-4c3a-a665-235a8bff906c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:38.617592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9ea43-ad0e-4c3a-a665-235a8bff906c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:38.661184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2803caf-effe-4d56-a59c-a1399188370b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:38.694217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9243c-d690-4a6c-ba04-258eb0b40809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:38.694979 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9243c-d690-4a6c-ba04-258eb0b40809 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.695291 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9243c-d690-4a6c-ba04-258eb0b40809 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.695572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9243c-d690-4a6c-ba04-258eb0b40809 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.696315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9243c-d690-4a6c-ba04-258eb0b40809 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:38.696624 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9243c-d690-4a6c-ba04-258eb0b40809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:38.698049 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9243c-d690-4a6c-ba04-258eb0b40809 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:38.705813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 40/76] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:15:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:38.706625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9243c-d690-4a6c-ba04-258eb0b40809 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:38.718874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 37/77] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:15:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:15:38.720060 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae77133-4115-422b-8ef2-26d93e6fc7a2 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.720355 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae77133-4115-422b-8ef2-26d93e6fc7a2 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.720635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae77133-4115-422b-8ef2-26d93e6fc7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.737192 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-8ae77133-4115-422b-8ef2-26d93e6fc7a2 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:38.738370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-8ae77133-4115-422b-8ef2-26d93e6fc7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:38.802664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9243c-d690-4a6c-ba04-258eb0b40809 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:38.807136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 38/78] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:38.809115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae77133-4115-422b-8ef2-26d93e6fc7a2 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:38.836348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f0c00-7a91-4ee6-8dc0-5e471ec6d5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:38.837024 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f0c00-7a91-4ee6-8dc0-5e471ec6d5f0 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.837306 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f0c00-7a91-4ee6-8dc0-5e471ec6d5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.837582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f0c00-7a91-4ee6-8dc0-5e471ec6d5f0 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.838332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f0c00-7a91-4ee6-8dc0-5e471ec6d5f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:38.838638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f0c00-7a91-4ee6-8dc0-5e471ec6d5f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:38.840055 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f0c00-7a91-4ee6-8dc0-5e471ec6d5f0 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:38.848551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 41/79] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:38.849347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f0c00-7a91-4ee6-8dc0-5e471ec6d5f0 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:38.906970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae77133-4115-422b-8ef2-26d93e6fc7a2 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:38.942183 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e6667-27e3-4097-802a-34a349ec96b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:38.942988 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e6667-27e3-4097-802a-34a349ec96b9 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.943311 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e6667-27e3-4097-802a-34a349ec96b9 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.943573 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f0c00-7a91-4ee6-8dc0-5e471ec6d5f0 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:38.943667 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e6667-27e3-4097-802a-34a349ec96b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.953499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e6667-27e3-4097-802a-34a349ec96b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-690e6667-27e3-4097-802a-34a349ec96b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:38.961324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:38.962314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:38.963123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:38.963123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:38.963123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:38.963123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:38.963123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:38.963123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:38.963123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:38.963123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:38.963123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:38.963123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:38.963123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:15:38.963123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:38.963123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 39/80] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:38.964196 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e6667-27e3-4097-802a-34a349ec96b9 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:38.977329 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809a04b-06c4-46d3-bc33-2e0681b78ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:38.978212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809a04b-06c4-46d3-bc33-2e0681b78ae2 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:38.978497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809a04b-06c4-46d3-bc33-2e0681b78ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:38.978776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809a04b-06c4-46d3-bc33-2e0681b78ae2 None admin] PATH_INFO: `/v3/users/416a6d6800844d94a22ac939f6f6ba2a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:38.980002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e809a04b-06c4-46d3-bc33-2e0681b78ae2 None admin] RBAC: Authorizing `identity:get_user(user_id=416a6d6800844d94a22ac939f6f6ba2a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:38.981445 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e809a04b-06c4-46d3-bc33-2e0681b78ae2 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:38.983076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 42/81] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:38 2022] GET /identity/v3/users/416a6d6800844d94a22ac939f6f6ba2a => generated 275 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:38.983867 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809a04b-06c4-46d3-bc33-2e0681b78ae2 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:39.063192 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e6667-27e3-4097-802a-34a349ec96b9 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:39.082730 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809a04b-06c4-46d3-bc33-2e0681b78ae2 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:39.095003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0929092-f55b-4389-8621-35d0c7fb2105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:39.095902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0929092-f55b-4389-8621-35d0c7fb2105 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:39.096183 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0929092-f55b-4389-8621-35d0c7fb2105 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:39.096459 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0929092-f55b-4389-8621-35d0c7fb2105 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:39.097643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0929092-f55b-4389-8621-35d0c7fb2105 None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:39.099790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0929092-f55b-4389-8621-35d0c7fb2105 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:39.101478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 40/82] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:38 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:39.102455 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0929092-f55b-4389-8621-35d0c7fb2105 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:39.116061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8d069-d861-48ad-ad14-e4eff1384d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:39.116859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8d069-d861-48ad-ad14-e4eff1384d48 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:39.117139 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8d069-d861-48ad-ad14-e4eff1384d48 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:39.117414 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8d069-d861-48ad-ad14-e4eff1384d48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:39.118567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8d069-d861-48ad-ad14-e4eff1384d48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:39.120030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8d069-d861-48ad-ad14-e4eff1384d48 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:39.121633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 43/83] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:15:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:39.122391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8d069-d861-48ad-ad14-e4eff1384d48 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:39.196033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0929092-f55b-4389-8621-35d0c7fb2105 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:39.215879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8d069-d861-48ad-ad14-e4eff1384d48 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:39.227691 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78451749-733f-4bc8-8045-8c649ee16527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:39.228571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78451749-733f-4bc8-8045-8c649ee16527 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:39.228856 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78451749-733f-4bc8-8045-8c649ee16527 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:39.229133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78451749-733f-4bc8-8045-8c649ee16527 None admin] PATH_INFO: `/v3/users/a379c334e6cd446c80b18fda6ce9162e` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:39.230332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78451749-733f-4bc8-8045-8c649ee16527 None admin] RBAC: Authorizing `identity:get_user(user_id=a379c334e6cd446c80b18fda6ce9162e)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:39.231805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78451749-733f-4bc8-8045-8c649ee16527 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:39.233429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 41/84] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:39 2022] GET /identity/v3/users/a379c334e6cd446c80b18fda6ce9162e => generated 247 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:39.234192 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78451749-733f-4bc8-8045-8c649ee16527 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:39.247695 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7a00c-78a3-4879-9669-27ff2f84e2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:39.248357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7a00c-78a3-4879-9669-27ff2f84e2f8 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:39.248638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7a00c-78a3-4879-9669-27ff2f84e2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:39.248922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7a00c-78a3-4879-9669-27ff2f84e2f8 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:39.249670 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7a00c-78a3-4879-9669-27ff2f84e2f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:39.249961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7a00c-78a3-4879-9669-27ff2f84e2f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:39.251346 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7a00c-78a3-4879-9669-27ff2f84e2f8 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:39.259266 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 44/85] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:15:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:39.260036 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7a00c-78a3-4879-9669-27ff2f84e2f8 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:39.327137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78451749-733f-4bc8-8045-8c649ee16527 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:39.352456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7a00c-78a3-4879-9669-27ff2f84e2f8 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:39.359637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f320bb-7235-4dbd-85bc-30fb1c25e85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:39.360297 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f320bb-7235-4dbd-85bc-30fb1c25e85e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:39.360587 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f320bb-7235-4dbd-85bc-30fb1c25e85e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:39.360868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f320bb-7235-4dbd-85bc-30fb1c25e85e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:39.383457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aeb83e-9b74-4b27-a74a-82ff16720514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:39.384104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aeb83e-9b74-4b27-a74a-82ff16720514 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:39.384389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aeb83e-9b74-4b27-a74a-82ff16720514 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:39.384665 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aeb83e-9b74-4b27-a74a-82ff16720514 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:39.385489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aeb83e-9b74-4b27-a74a-82ff16720514 None admin] RBAC: Adding query filter params (role.id=e1dddf0b88ee4578822ce51a2aba6760, scope.project.id=36617c1516b14f15a3b4a35692ff654f, user.id=416a6d6800844d94a22ac939f6f6ba2a) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:39.385799 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aeb83e-9b74-4b27-a74a-82ff16720514 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:39.387241 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aeb83e-9b74-4b27-a74a-82ff16720514 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:39.392408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 45/86] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:15:39 2022] GET /identity/v3/role_assignments?user.id=416a6d6800844d94a22ac939f6f6ba2a&scope.project.id=36617c1516b14f15a3b4a35692ff654f&role.id=e1dddf0b88ee4578822ce51a2aba6760 => generated 259 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:39.407507 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aeb83e-9b74-4b27-a74a-82ff16720514 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:39.450259 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f320bb-7235-4dbd-85bc-30fb1c25e85e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:39.451693 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f320bb-7235-4dbd-85bc-30fb1c25e85e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:39.509086 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aeb83e-9b74-4b27-a74a-82ff16720514 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:39.544817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c3026-929e-47a3-a249-5b680c306d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:39.545501 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c3026-929e-47a3-a249-5b680c306d13 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:39.545784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c3026-929e-47a3-a249-5b680c306d13 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:39.546061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c3026-929e-47a3-a249-5b680c306d13 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:39.546782 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c3026-929e-47a3-a249-5b680c306d13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:39.548290 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c3026-929e-47a3-a249-5b680c306d13 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:39.559537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 42/87] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:15:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:15:39.559537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f320bb-7235-4dbd-85bc-30fb1c25e85e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:39.579332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 46/88] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:15:39 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:15:39.579332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c3026-929e-47a3-a249-5b680c306d13 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:39.667120 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f320bb-7235-4dbd-85bc-30fb1c25e85e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:39.689752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c3026-929e-47a3-a249-5b680c306d13 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:39.703657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09636bf2-143d-413f-bf4a-68656eb03505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:39.704248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09636bf2-143d-413f-bf4a-68656eb03505 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:39.704536 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09636bf2-143d-413f-bf4a-68656eb03505 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:39.704817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09636bf2-143d-413f-bf4a-68656eb03505 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:39.728033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff22c4-74e7-46e6-9f3a-e32abdb32179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:39.729104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff22c4-74e7-46e6-9f3a-e32abdb32179 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:39.729495 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff22c4-74e7-46e6-9f3a-e32abdb32179 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:39.729879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff22c4-74e7-46e6-9f3a-e32abdb32179 None admin] PATH_INFO: `/v3/users/416a6d6800844d94a22ac939f6f6ba2a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:39.731336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff22c4-74e7-46e6-9f3a-e32abdb32179 None admin] RBAC: Authorizing `identity:get_user(user_id=416a6d6800844d94a22ac939f6f6ba2a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:39.734052 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff22c4-74e7-46e6-9f3a-e32abdb32179 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:39.736192 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 47/89] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:39 2022] GET /identity/v3/users/416a6d6800844d94a22ac939f6f6ba2a => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:39.737127 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff22c4-74e7-46e6-9f3a-e32abdb32179 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:39.778469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09636bf2-143d-413f-bf4a-68656eb03505 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:39.780229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09636bf2-143d-413f-bf4a-68656eb03505 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:39.849853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff22c4-74e7-46e6-9f3a-e32abdb32179 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:39.898796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38f2d4-963d-406c-962e-1e82adbe00d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:39.899708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38f2d4-963d-406c-962e-1e82adbe00d4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:39.899990 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38f2d4-963d-406c-962e-1e82adbe00d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:39.900469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38f2d4-963d-406c-962e-1e82adbe00d4 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:39.901776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38f2d4-963d-406c-962e-1e82adbe00d4 None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:39.903574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38f2d4-963d-406c-962e-1e82adbe00d4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:39.905683 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 48/90] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:39 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:39.906572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38f2d4-963d-406c-962e-1e82adbe00d4 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:39.909818 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 43/91] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:15:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:15:39.915046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09636bf2-143d-413f-bf4a-68656eb03505 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:40.036261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38f2d4-963d-406c-962e-1e82adbe00d4 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:40.038228 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09636bf2-143d-413f-bf4a-68656eb03505 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:40.070265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82d61d-f319-4137-b748-ced33f03737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:40.070925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82d61d-f319-4137-b748-ced33f03737c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:40.071227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82d61d-f319-4137-b748-ced33f03737c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:40.071508 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82d61d-f319-4137-b748-ced33f03737c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:40.081026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4318e1e-b5d9-4567-9afd-099048d8e725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:40.081702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4318e1e-b5d9-4567-9afd-099048d8e725 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:40.081996 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4318e1e-b5d9-4567-9afd-099048d8e725 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:40.082310 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4318e1e-b5d9-4567-9afd-099048d8e725 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:40.084995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4318e1e-b5d9-4567-9afd-099048d8e725 None admin] RBAC: Adding query filter params (role.id=9d45ca30c3584deabd69f2cd196ae5cc, scope.project.id=36617c1516b14f15a3b4a35692ff654f, user.id=416a6d6800844d94a22ac939f6f6ba2a) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:40.084995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4318e1e-b5d9-4567-9afd-099048d8e725 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:40.087764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4318e1e-b5d9-4567-9afd-099048d8e725 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:40.094873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 44/92] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:15:39 2022] GET /identity/v3/role_assignments?user.id=416a6d6800844d94a22ac939f6f6ba2a&scope.project.id=36617c1516b14f15a3b4a35692ff654f&role.id=9d45ca30c3584deabd69f2cd196ae5cc => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:40.096178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4318e1e-b5d9-4567-9afd-099048d8e725 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:40.212348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4318e1e-b5d9-4567-9afd-099048d8e725 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:40.220349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82d61d-f319-4137-b748-ced33f03737c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:40.221832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82d61d-f319-4137-b748-ced33f03737c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:40.252314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b96093-488b-477e-b5cc-23843bd43c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:40.253743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b96093-488b-477e-b5cc-23843bd43c25 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:40.254039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b96093-488b-477e-b5cc-23843bd43c25 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:40.254931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b96093-488b-477e-b5cc-23843bd43c25 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:40.256223 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b96093-488b-477e-b5cc-23843bd43c25 None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:40.258220 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b96093-488b-477e-b5cc-23843bd43c25 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:40.260216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 45/93] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:40 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:40.266796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b96093-488b-477e-b5cc-23843bd43c25 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:40.385996 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b96093-488b-477e-b5cc-23843bd43c25 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:40.424109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 49/94] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:15:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:15:40.430156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82d61d-f319-4137-b748-ced33f03737c None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:40.449350 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065006f4-048f-45f8-9545-cdcf438be1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:40.450069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065006f4-048f-45f8-9545-cdcf438be1d5 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:40.450361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065006f4-048f-45f8-9545-cdcf438be1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:40.450690 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065006f4-048f-45f8-9545-cdcf438be1d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:40.451569 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065006f4-048f-45f8-9545-cdcf438be1d5 None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=36617c1516b14f15a3b4a35692ff654f, user.id=a379c334e6cd446c80b18fda6ce9162e) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:40.451885 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065006f4-048f-45f8-9545-cdcf438be1d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:40.453721 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065006f4-048f-45f8-9545-cdcf438be1d5 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:40.460413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 46/95] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:15:40 2022] GET /identity/v3/role_assignments?user.id=a379c334e6cd446c80b18fda6ce9162e&scope.project.id=36617c1516b14f15a3b4a35692ff654f&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:15:40.618765 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82d61d-f319-4137-b748-ced33f03737c None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:40.653233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c1b15-75e9-4f32-8b86-c8e5be39d8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:40.654123 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c1b15-75e9-4f32-8b86-c8e5be39d8be None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:40.654406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c1b15-75e9-4f32-8b86-c8e5be39d8be None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:40.654685 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c1b15-75e9-4f32-8b86-c8e5be39d8be None admin] PATH_INFO: `/v3/projects/b64c14b29bb542b0b809d69c7c52b0be` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:40.655926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c1b15-75e9-4f32-8b86-c8e5be39d8be None admin] RBAC: Authorizing `identity:get_project(project_id=b64c14b29bb542b0b809d69c7c52b0be)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:40.657412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c1b15-75e9-4f32-8b86-c8e5be39d8be None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:40.659534 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 50/96] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:40 2022] GET /identity/v3/projects/b64c14b29bb542b0b809d69c7c52b0be => generated 312 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:40.665642 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065006f4-048f-45f8-9545-cdcf438be1d5 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:40.770604 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065006f4-048f-45f8-9545-cdcf438be1d5 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:40.805431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6ef79-77f1-46cb-82c4-578ff7d1da66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:40.806409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6ef79-77f1-46cb-82c4-578ff7d1da66 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:40.806873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6ef79-77f1-46cb-82c4-578ff7d1da66 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:40.807358 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6ef79-77f1-46cb-82c4-578ff7d1da66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:40.808453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6ef79-77f1-46cb-82c4-578ff7d1da66 None admin] RBAC: Adding query filter params (role.id=e1dddf0b88ee4578822ce51a2aba6760, scope.project.id=b64c14b29bb542b0b809d69c7c52b0be, user.id=416a6d6800844d94a22ac939f6f6ba2a) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:40.808951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6ef79-77f1-46cb-82c4-578ff7d1da66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:40.810616 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6ef79-77f1-46cb-82c4-578ff7d1da66 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:40.817097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 47/97] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:15:40 2022] GET /identity/v3/role_assignments?user.id=416a6d6800844d94a22ac939f6f6ba2a&scope.project.id=b64c14b29bb542b0b809d69c7c52b0be&role.id=e1dddf0b88ee4578822ce51a2aba6760 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:42.855444 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cabd2-184f-4e9b-b8ba-50ef360b1b79 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:42.856348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cabd2-184f-4e9b-b8ba-50ef360b1b79 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:42.856798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cabd2-184f-4e9b-b8ba-50ef360b1b79 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:42.858568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 51/98] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:42.864255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cead9-baaf-40fb-a5b5-946badc747b9 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:42.865036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cead9-baaf-40fb-a5b5-946badc747b9 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:42.865494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cead9-baaf-40fb-a5b5-946badc747b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:42.889288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-b60cead9-baaf-40fb-a5b5-946badc747b9 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:42.890719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b60cead9-baaf-40fb-a5b5-946badc747b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:42.981492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 48/99] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:42.991113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc331b-0a06-45e5-a702-855ed1946eb7 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:42.991434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc331b-0a06-45e5-a702-855ed1946eb7 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:42.991721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc331b-0a06-45e5-a702-855ed1946eb7 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:42.993080 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 52/100] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:43.005003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b35ef-a618-4f25-83d1-fccf49c2450a None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:43.005298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b35ef-a618-4f25-83d1-fccf49c2450a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:43.005580 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b35ef-a618-4f25-83d1-fccf49c2450a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:43.028532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-e82b35ef-a618-4f25-83d1-fccf49c2450a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:43.029719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e82b35ef-a618-4f25-83d1-fccf49c2450a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:43.044887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff221f50-cb76-4b3c-b4da-cf96327e7f03 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:43.045184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff221f50-cb76-4b3c-b4da-cf96327e7f03 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:43.045468 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff221f50-cb76-4b3c-b4da-cf96327e7f03 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:43.047078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 53/101] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:43.060780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6737b2-ecf5-4934-bef0-f41c659fcf3b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:43.061564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6737b2-ecf5-4934-bef0-f41c659fcf3b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:43.062377 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6737b2-ecf5-4934-bef0-f41c659fcf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:43.092243 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-ed6737b2-ecf5-4934-bef0-f41c659fcf3b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:43.093646 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ed6737b2-ecf5-4934-bef0-f41c659fcf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:43.117990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 49/102] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:43.119140 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b35ef-a618-4f25-83d1-fccf49c2450a None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:43.195138 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 54/103] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:15:43.197276 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0726f7-cb68-41c1-b461-3e879c7cae5b None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:43.197707 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0726f7-cb68-41c1-b461-3e879c7cae5b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:43.198122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0726f7-cb68-41c1-b461-3e879c7cae5b None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:43.200314 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 55/104] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:43.201504 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0726f7-cb68-41c1-b461-3e879c7cae5b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:43.218706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b35ef-a618-4f25-83d1-fccf49c2450a None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:43.255233 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea390f-84be-4e79-8bbb-66e56f644acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:43.256306 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea390f-84be-4e79-8bbb-66e56f644acf None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:43.256762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea390f-84be-4e79-8bbb-66e56f644acf None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:43.257167 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea390f-84be-4e79-8bbb-66e56f644acf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:43.267495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea390f-84be-4e79-8bbb-66e56f644acf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-41ea390f-84be-4e79-8bbb-66e56f644acf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:43.275507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:43.276349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:43.277094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:43.277094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:43.277094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:43.277094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:43.277094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:43.277094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:43.277094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:43.277094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:43.277094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:43.277094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:43.277094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:15:43.277094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:43.279731 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 50/105] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:43.281229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650d98c-7f4f-40d5-924e-ae706232ae2e None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:43.281709 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650d98c-7f4f-40d5-924e-ae706232ae2e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:43.282175 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650d98c-7f4f-40d5-924e-ae706232ae2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:43.298580 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-c650d98c-7f4f-40d5-924e-ae706232ae2e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:43.300077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c650d98c-7f4f-40d5-924e-ae706232ae2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:43.320161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0726f7-cb68-41c1-b461-3e879c7cae5b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:43.367354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6889b8b-59b9-47f8-90bc-049a8d1eca03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:43.368240 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6889b8b-59b9-47f8-90bc-049a8d1eca03 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:43.368469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6889b8b-59b9-47f8-90bc-049a8d1eca03 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:43.368749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6889b8b-59b9-47f8-90bc-049a8d1eca03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:43.370685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 51/106] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:43.371855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650d98c-7f4f-40d5-924e-ae706232ae2e None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:43.381592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6889b8b-59b9-47f8-90bc-049a8d1eca03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-b6889b8b-59b9-47f8-90bc-049a8d1eca03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:43.391420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:43.392313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:43.393072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:43.393072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:43.393072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:43.393072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:43.393072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:43.393072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:43.393072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:43.393072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:43.393072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:43.393072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:43.393072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:15:43.393072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:15:43.393072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 56/107] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:43.393681 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6889b8b-59b9-47f8-90bc-049a8d1eca03 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:43.478135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650d98c-7f4f-40d5-924e-ae706232ae2e None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:43.512614 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b0495-1320-4b95-a73f-3fe2070c3960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:43.513699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b0495-1320-4b95-a73f-3fe2070c3960 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:43.514150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b0495-1320-4b95-a73f-3fe2070c3960 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:43.514588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b0495-1320-4b95-a73f-3fe2070c3960 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:43.516028 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b0495-1320-4b95-a73f-3fe2070c3960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:43.517645 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b0495-1320-4b95-a73f-3fe2070c3960 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:43.519803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 52/108] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:15:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:43.520829 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b0495-1320-4b95-a73f-3fe2070c3960 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:43.521499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6889b8b-59b9-47f8-90bc-049a8d1eca03 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:43.560438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222570b6-777f-4a8c-b5a7-6a951e7a52e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:43.561522 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222570b6-777f-4a8c-b5a7-6a951e7a52e1 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:43.562045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222570b6-777f-4a8c-b5a7-6a951e7a52e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:43.562524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222570b6-777f-4a8c-b5a7-6a951e7a52e1 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:43.563546 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222570b6-777f-4a8c-b5a7-6a951e7a52e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:43.564044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222570b6-777f-4a8c-b5a7-6a951e7a52e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:43.565666 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222570b6-777f-4a8c-b5a7-6a951e7a52e1 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:43.574518 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 57/109] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:15:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:43.575679 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222570b6-777f-4a8c-b5a7-6a951e7a52e1 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:43.645553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b0495-1320-4b95-a73f-3fe2070c3960 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:43.679910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222570b6-777f-4a8c-b5a7-6a951e7a52e1 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:43.696955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390690e-650b-420f-aebf-2d4d3b0b04a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:43.698072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390690e-650b-420f-aebf-2d4d3b0b04a4 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:43.698366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390690e-650b-420f-aebf-2d4d3b0b04a4 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:43.698650 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390690e-650b-420f-aebf-2d4d3b0b04a4 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:43.699487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390690e-650b-420f-aebf-2d4d3b0b04a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:43.699801 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390690e-650b-420f-aebf-2d4d3b0b04a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:43.701373 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390690e-650b-420f-aebf-2d4d3b0b04a4 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:43.713068 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 53/110] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:15:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:43.713882 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390690e-650b-420f-aebf-2d4d3b0b04a4 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:43.720216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c882f-0b64-45cd-9260-5f2042aebbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:43.721359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c882f-0b64-45cd-9260-5f2042aebbdb None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:43.721836 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c882f-0b64-45cd-9260-5f2042aebbdb None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:43.722304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c882f-0b64-45cd-9260-5f2042aebbdb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:43.734317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c882f-0b64-45cd-9260-5f2042aebbdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-b09c882f-0b64-45cd-9260-5f2042aebbdb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:43.743650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:43.744737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:43.745515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:43.745515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:43.745515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:43.745515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:43.745515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:43.745515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:43.745515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:43.745515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:43.745515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:43.745515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:43.745515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:15:43.745515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:15:43.748263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 58/111] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:15:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:43.749488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c882f-0b64-45cd-9260-5f2042aebbdb None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:43.834961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390690e-650b-420f-aebf-2d4d3b0b04a4 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:43.860094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c882f-0b64-45cd-9260-5f2042aebbdb None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:43.871419 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e9976-8329-4e40-8d42-705b213c1143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:43.872091 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e9976-8329-4e40-8d42-705b213c1143 None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:43.872376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e9976-8329-4e40-8d42-705b213c1143 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:43.872652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e9976-8329-4e40-8d42-705b213c1143 None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:43.873363 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e9976-8329-4e40-8d42-705b213c1143 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:43.874801 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e9976-8329-4e40-8d42-705b213c1143 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:43.900265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71909c2-8419-4c86-91b1-4483c92421e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:43.901209 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 54/112] 10.222.0.25 () {60 vars in 1141 bytes} [Mon May 9 18:15:43 2022] POST /identity/v3/projects => generated 302 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:15:43.901878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e9976-8329-4e40-8d42-705b213c1143 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:43.911470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71909c2-8419-4c86-91b1-4483c92421e0 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:43.911872 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71909c2-8419-4c86-91b1-4483c92421e0 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:43.912198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71909c2-8419-4c86-91b1-4483c92421e0 None admin] PATH_INFO: `/v3/users/fda72ad9ae3049c08f96773911da779a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:43.928420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71909c2-8419-4c86-91b1-4483c92421e0 None admin] RBAC: Authorizing `identity:get_user(user_id=fda72ad9ae3049c08f96773911da779a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:43.930230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71909c2-8419-4c86-91b1-4483c92421e0 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:43.932484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 59/113] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:43 2022] GET /identity/v3/users/fda72ad9ae3049c08f96773911da779a => generated 289 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:43.933412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71909c2-8419-4c86-91b1-4483c92421e0 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:44.033949 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e9976-8329-4e40-8d42-705b213c1143 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:44.043653 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71909c2-8419-4c86-91b1-4483c92421e0 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:44.074329 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ebac8-d012-4795-9728-8ffddd4b8410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:44.075245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ebac8-d012-4795-9728-8ffddd4b8410 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.075541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ebac8-d012-4795-9728-8ffddd4b8410 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.075820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ebac8-d012-4795-9728-8ffddd4b8410 None admin] PATH_INFO: `/v3/users/416a6d6800844d94a22ac939f6f6ba2a` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.077025 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ebac8-d012-4795-9728-8ffddd4b8410 None admin] RBAC: Authorizing `identity:get_user(user_id=416a6d6800844d94a22ac939f6f6ba2a)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:44.078479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ebac8-d012-4795-9728-8ffddd4b8410 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:44.079583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4d76c-f947-4884-866d-4f85d53598c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:44.080256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d76c-f947-4884-866d-4f85d53598c1 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.080540 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d76c-f947-4884-866d-4f85d53598c1 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.080816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d76c-f947-4884-866d-4f85d53598c1 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.081215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 55/114] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:43 2022] GET /identity/v3/users/416a6d6800844d94a22ac939f6f6ba2a => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:44.081570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4d76c-f947-4884-866d-4f85d53598c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:44.081913 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4d76c-f947-4884-866d-4f85d53598c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:44.083169 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcc571-6406-4d27-aeeb-16a389413e36 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.083485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcc571-6406-4d27-aeeb-16a389413e36 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.083770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcc571-6406-4d27-aeeb-16a389413e36 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.083870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4d76c-f947-4884-866d-4f85d53598c1 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:44.085188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 56/115] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:44.085823 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcc571-6406-4d27-aeeb-16a389413e36 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:44.093277 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 60/116] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:15:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:44.094363 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff838d3b-4e9f-4ff2-96d3-d0d160918506 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.094363 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff838d3b-4e9f-4ff2-96d3-d0d160918506 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.094523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff838d3b-4e9f-4ff2-96d3-d0d160918506 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.095853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 61/117] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:44.097135 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c82956-371d-4eff-9913-b6a7b31bcefe None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.097135 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c82956-371d-4eff-9913-b6a7b31bcefe None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.097307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c82956-371d-4eff-9913-b6a7b31bcefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.115400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-68c82956-371d-4eff-9913-b6a7b31bcefe None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:44.116600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-68c82956-371d-4eff-9913-b6a7b31bcefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:44.185383 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 62/118] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:44.186530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c82956-371d-4eff-9913-b6a7b31bcefe None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:44.206468 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcc571-6406-4d27-aeeb-16a389413e36 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:44.239663 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775c081-eb0e-4998-b923-6913b5e18d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:44.240319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775c081-eb0e-4998-b923-6913b5e18d64 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.240631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775c081-eb0e-4998-b923-6913b5e18d64 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.240873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775c081-eb0e-4998-b923-6913b5e18d64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.293204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c82956-371d-4eff-9913-b6a7b31bcefe None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:44.306475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775c081-eb0e-4998-b923-6913b5e18d64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:44.308127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775c081-eb0e-4998-b923-6913b5e18d64 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:44.326197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a8799-3824-496f-adf3-08491a489889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:44.327054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a8799-3824-496f-adf3-08491a489889 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.327370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a8799-3824-496f-adf3-08491a489889 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.327646 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a8799-3824-496f-adf3-08491a489889 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.328815 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a8799-3824-496f-adf3-08491a489889 None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:44.330315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a8799-3824-496f-adf3-08491a489889 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:44.332090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 63/119] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:44 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:44.333464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a8799-3824-496f-adf3-08491a489889 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:44.417623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 57/120] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:15:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:15:44.418913 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2804449-092b-4b12-9c3b-d7bba1335105 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.419224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2804449-092b-4b12-9c3b-d7bba1335105 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.419505 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2804449-092b-4b12-9c3b-d7bba1335105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.437243 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a8799-3824-496f-adf3-08491a489889 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:44.439899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-e2804449-092b-4b12-9c3b-d7bba1335105 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:44.441056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e2804449-092b-4b12-9c3b-d7bba1335105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:44.471020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce3d0f-bbeb-4e95-b2fc-353b43a44400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:44.471928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce3d0f-bbeb-4e95-b2fc-353b43a44400 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.472212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce3d0f-bbeb-4e95-b2fc-353b43a44400 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.472489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce3d0f-bbeb-4e95-b2fc-353b43a44400 None admin] PATH_INFO: `/v3/users/416a6d6800844d94a22ac939f6f6ba2a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.473741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce3d0f-bbeb-4e95-b2fc-353b43a44400 None admin] RBAC: Authorizing `identity:get_user(user_id=416a6d6800844d94a22ac939f6f6ba2a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:44.475223 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce3d0f-bbeb-4e95-b2fc-353b43a44400 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:44.476894 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 64/121] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:44 2022] GET /identity/v3/users/416a6d6800844d94a22ac939f6f6ba2a => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:44.477645 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce3d0f-bbeb-4e95-b2fc-353b43a44400 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:44.511988 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 58/122] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:44.512651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2804449-092b-4b12-9c3b-d7bba1335105 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:44.584947 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce3d0f-bbeb-4e95-b2fc-353b43a44400 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:44.609367 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2804449-092b-4b12-9c3b-d7bba1335105 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:44.619106 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7670f-194f-4136-b5c9-cdbe53bbb7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:44.619940 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7670f-194f-4136-b5c9-cdbe53bbb7a0 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.620223 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7670f-194f-4136-b5c9-cdbe53bbb7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.620499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7670f-194f-4136-b5c9-cdbe53bbb7a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.631344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7670f-194f-4136-b5c9-cdbe53bbb7a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-6fd7670f-194f-4136-b5c9-cdbe53bbb7a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:44.639408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:44.640250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:44.641004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:44.641004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:44.641004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:44.641004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:44.641004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:44.641004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:44.641004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:44.641004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:44.641004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:44.641004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:44.641004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:15:44.641004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:15:44.641004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 65/123] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:44.641643 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7670f-194f-4136-b5c9-cdbe53bbb7a0 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:44.649119 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28b3ff-fe4f-475e-97b8-4f117b488b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:44.650016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28b3ff-fe4f-475e-97b8-4f117b488b7f None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.650295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28b3ff-fe4f-475e-97b8-4f117b488b7f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.650572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28b3ff-fe4f-475e-97b8-4f117b488b7f None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f/users/416a6d6800844d94a22ac939f6f6ba2a/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.652522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28b3ff-fe4f-475e-97b8-4f117b488b7f None admin] RBAC: Authorizing `identity:create_grant(project_id=36617c1516b14f15a3b4a35692ff654f, user_id=416a6d6800844d94a22ac939f6f6ba2a, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:44.653956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28b3ff-fe4f-475e-97b8-4f117b488b7f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:44.668114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 59/124] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:15:44 2022] PUT /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f/users/416a6d6800844d94a22ac939f6f6ba2a/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:15:44.668933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28b3ff-fe4f-475e-97b8-4f117b488b7f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:44.767282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7670f-194f-4136-b5c9-cdbe53bbb7a0 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:44.795926 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28b3ff-fe4f-475e-97b8-4f117b488b7f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:44.813246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd8c6a-ce29-4108-8736-10611ceaef0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:44.814928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd8c6a-ce29-4108-8736-10611ceaef0c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.815427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd8c6a-ce29-4108-8736-10611ceaef0c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.815882 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd8c6a-ce29-4108-8736-10611ceaef0c None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.817967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd8c6a-ce29-4108-8736-10611ceaef0c None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:44.820654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd8c6a-ce29-4108-8736-10611ceaef0c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:44.828179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 66/125] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:44 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:44.828355 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd8c6a-ce29-4108-8736-10611ceaef0c None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:44.838353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf13519-9657-45d6-b929-bf300f18f26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:44.839282 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf13519-9657-45d6-b929-bf300f18f26c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.839570 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf13519-9657-45d6-b929-bf300f18f26c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.839845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf13519-9657-45d6-b929-bf300f18f26c None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.841142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf13519-9657-45d6-b929-bf300f18f26c None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:44.842651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf13519-9657-45d6-b929-bf300f18f26c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:44.844761 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 60/126] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:44 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:44.845666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf13519-9657-45d6-b929-bf300f18f26c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:44.936062 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd8c6a-ce29-4108-8736-10611ceaef0c None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:44.950386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf13519-9657-45d6-b929-bf300f18f26c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:44.973984 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984827b3-7991-4805-8d8a-1f47a94b8078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:44.973984 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984827b3-7991-4805-8d8a-1f47a94b8078 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.973984 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984827b3-7991-4805-8d8a-1f47a94b8078 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.973984 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984827b3-7991-4805-8d8a-1f47a94b8078 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.983911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984827b3-7991-4805-8d8a-1f47a94b8078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:44.985512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698c49f-f2ac-41c2-b395-79fbd12cec70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:44.986197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698c49f-f2ac-41c2-b395-79fbd12cec70 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:44.986488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698c49f-f2ac-41c2-b395-79fbd12cec70 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:44.986768 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698c49f-f2ac-41c2-b395-79fbd12cec70 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:44.987551 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698c49f-f2ac-41c2-b395-79fbd12cec70 None admin] RBAC: Adding query filter params (name=member) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:44.987864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698c49f-f2ac-41c2-b395-79fbd12cec70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:44.989789 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698c49f-f2ac-41c2-b395-79fbd12cec70 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-984827b3-7991-4805-8d8a-1f47a94b8078 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:44.992145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:44.992972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:44.993739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:44.993739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:44.993739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:44.993739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:44.993739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:44.993739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:44.993739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:44.993739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:44.993739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:44.993739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:44.993739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:44.993739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:15:44.993739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 67/127] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:15:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:44.994361 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984827b3-7991-4805-8d8a-1f47a94b8078 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:45.003157 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 61/128] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:15:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:45.004044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698c49f-f2ac-41c2-b395-79fbd12cec70 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:45.095611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984827b3-7991-4805-8d8a-1f47a94b8078 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:45.104269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698c49f-f2ac-41c2-b395-79fbd12cec70 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:45.131198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25723ed5-1889-48ed-8efe-277e542b1245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:45.131873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25723ed5-1889-48ed-8efe-277e542b1245 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:45.132169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25723ed5-1889-48ed-8efe-277e542b1245 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:45.132450 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25723ed5-1889-48ed-8efe-277e542b1245 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:45.133274 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25723ed5-1889-48ed-8efe-277e542b1245 None admin] RBAC: Adding query filter params (role.id=3c0125c8cabb462197565509a20404dc, scope.project.id=36617c1516b14f15a3b4a35692ff654f, user.id=fda72ad9ae3049c08f96773911da779a) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:45.133590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25723ed5-1889-48ed-8efe-277e542b1245 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:45.135039 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25723ed5-1889-48ed-8efe-277e542b1245 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:45.141876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 68/129] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:15:44 2022] GET /identity/v3/role_assignments?user.id=fda72ad9ae3049c08f96773911da779a&scope.project.id=36617c1516b14f15a3b4a35692ff654f&role.id=3c0125c8cabb462197565509a20404dc => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:45.142847 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25723ed5-1889-48ed-8efe-277e542b1245 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:45.144575 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e19406-236a-43ee-bfaf-7e5ca888763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:45.145757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e19406-236a-43ee-bfaf-7e5ca888763f None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:45.146273 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e19406-236a-43ee-bfaf-7e5ca888763f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:45.146726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e19406-236a-43ee-bfaf-7e5ca888763f None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f/users/416a6d6800844d94a22ac939f6f6ba2a/roles/9d45ca30c3584deabd69f2cd196ae5cc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:45.149353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e19406-236a-43ee-bfaf-7e5ca888763f None admin] RBAC: Authorizing `identity:create_grant(project_id=36617c1516b14f15a3b4a35692ff654f, user_id=416a6d6800844d94a22ac939f6f6ba2a, role_id=9d45ca30c3584deabd69f2cd196ae5cc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:45.151039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e19406-236a-43ee-bfaf-7e5ca888763f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:45.173978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 62/130] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:15:45 2022] PUT /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f/users/416a6d6800844d94a22ac939f6f6ba2a/roles/9d45ca30c3584deabd69f2cd196ae5cc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:15:45.175281 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e19406-236a-43ee-bfaf-7e5ca888763f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:45.294593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e19406-236a-43ee-bfaf-7e5ca888763f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:45.305925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25723ed5-1889-48ed-8efe-277e542b1245 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:45.332590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670be8ae-cf31-4c56-b515-78462e18dca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:45.333814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670be8ae-cf31-4c56-b515-78462e18dca7 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:45.334275 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670be8ae-cf31-4c56-b515-78462e18dca7 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:45.334791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670be8ae-cf31-4c56-b515-78462e18dca7 None admin] PATH_INFO: `/v3/users/416a6d6800844d94a22ac939f6f6ba2a` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:45.336376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670be8ae-cf31-4c56-b515-78462e18dca7 None admin] RBAC: Authorizing `identity:get_user(user_id=416a6d6800844d94a22ac939f6f6ba2a)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:45.338162 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670be8ae-cf31-4c56-b515-78462e18dca7 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:45.340155 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 63/131] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:45 2022] GET /identity/v3/users/416a6d6800844d94a22ac939f6f6ba2a => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:45.349110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670be8ae-cf31-4c56-b515-78462e18dca7 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:45.397407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aa206-2f15-4708-a9a6-d1ac5560f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:45.398101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aa206-2f15-4708-a9a6-d1ac5560f601 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:45.398383 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aa206-2f15-4708-a9a6-d1ac5560f601 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:45.398666 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aa206-2f15-4708-a9a6-d1ac5560f601 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:45.399445 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685aa206-2f15-4708-a9a6-d1ac5560f601 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:45.399972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685aa206-2f15-4708-a9a6-d1ac5560f601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:45.401395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685aa206-2f15-4708-a9a6-d1ac5560f601 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:45.425531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 69/132] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:45.431910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aa206-2f15-4708-a9a6-d1ac5560f601 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:45.553400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670be8ae-cf31-4c56-b515-78462e18dca7 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:45.592081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335d85a-3845-419f-9798-98c0c09194c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:45.593005 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335d85a-3845-419f-9798-98c0c09194c6 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:45.593290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335d85a-3845-419f-9798-98c0c09194c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:45.593566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335d85a-3845-419f-9798-98c0c09194c6 None admin] PATH_INFO: `/v3/projects/b64c14b29bb542b0b809d69c7c52b0be` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:45.596891 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aa206-2f15-4708-a9a6-d1ac5560f601 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:45.600054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335d85a-3845-419f-9798-98c0c09194c6 None admin] RBAC: Authorizing `identity:get_project(project_id=b64c14b29bb542b0b809d69c7c52b0be)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:45.601468 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335d85a-3845-419f-9798-98c0c09194c6 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:45.603255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 64/133] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:45 2022] GET /identity/v3/projects/b64c14b29bb542b0b809d69c7c52b0be => generated 312 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:45.608820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335d85a-3845-419f-9798-98c0c09194c6 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:45.633528 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c43cd-8d22-45e3-9d01-a106227859cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:45.634417 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c43cd-8d22-45e3-9d01-a106227859cf None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:45.634704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c43cd-8d22-45e3-9d01-a106227859cf None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:45.635133 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c43cd-8d22-45e3-9d01-a106227859cf None admin] PATH_INFO: `/v3/users/a379c334e6cd446c80b18fda6ce9162e` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:45.636543 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c43cd-8d22-45e3-9d01-a106227859cf None admin] RBAC: Authorizing `identity:get_user(user_id=a379c334e6cd446c80b18fda6ce9162e)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:45.637992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c43cd-8d22-45e3-9d01-a106227859cf None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:45.640384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 70/134] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:45 2022] GET /identity/v3/users/a379c334e6cd446c80b18fda6ce9162e => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:15:45.647958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c43cd-8d22-45e3-9d01-a106227859cf None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:45.797400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c43cd-8d22-45e3-9d01-a106227859cf None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:45.838846 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335d85a-3845-419f-9798-98c0c09194c6 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:45.844197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d880c6-ffbc-475d-99d7-053b8434e90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:45.845155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d880c6-ffbc-475d-99d7-053b8434e90d None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:45.845509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d880c6-ffbc-475d-99d7-053b8434e90d None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:45.846809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d880c6-ffbc-475d-99d7-053b8434e90d None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:45.849736 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d880c6-ffbc-475d-99d7-053b8434e90d None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:45.852722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d880c6-ffbc-475d-99d7-053b8434e90d None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:45.855814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 71/135] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:45 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:45.861363 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d880c6-ffbc-475d-99d7-053b8434e90d None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:45.899883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ed790-b20c-4267-b0ec-0560f3bc2489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:45.901120 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed790-b20c-4267-b0ec-0560f3bc2489 None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:45.901611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed790-b20c-4267-b0ec-0560f3bc2489 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:45.902077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed790-b20c-4267-b0ec-0560f3bc2489 None admin] PATH_INFO: `/v3/projects/b64c14b29bb542b0b809d69c7c52b0be/users/416a6d6800844d94a22ac939f6f6ba2a/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:45.904326 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ed790-b20c-4267-b0ec-0560f3bc2489 None admin] RBAC: Authorizing `identity:create_grant(project_id=b64c14b29bb542b0b809d69c7c52b0be, user_id=416a6d6800844d94a22ac939f6f6ba2a, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:45.905977 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ed790-b20c-4267-b0ec-0560f3bc2489 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:45.929179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 65/136] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:15:45 2022] PUT /identity/v3/projects/b64c14b29bb542b0b809d69c7c52b0be/users/416a6d6800844d94a22ac939f6f6ba2a/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:15:45.982311 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d880c6-ffbc-475d-99d7-053b8434e90d None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:46.023159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257c97b-9610-4773-a72d-561de03084b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:46.024074 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257c97b-9610-4773-a72d-561de03084b7 None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:46.024359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257c97b-9610-4773-a72d-561de03084b7 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:46.024641 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257c97b-9610-4773-a72d-561de03084b7 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f/users/a379c334e6cd446c80b18fda6ce9162e/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:46.027643 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257c97b-9610-4773-a72d-561de03084b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=36617c1516b14f15a3b4a35692ff654f, user_id=a379c334e6cd446c80b18fda6ce9162e, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:46.029084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257c97b-9610-4773-a72d-561de03084b7 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:46.046178 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 72/137] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:15:45 2022] PUT /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f/users/a379c334e6cd446c80b18fda6ce9162e/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:15:46.789919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf310c-26af-4a6e-bd07-b9efdd884da8 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:46.789919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf310c-26af-4a6e-bd07-b9efdd884da8 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:46.789919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf310c-26af-4a6e-bd07-b9efdd884da8 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:46.789919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 66/138] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:46.807993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc1f7b-7613-430f-9521-10751f3546d5 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:46.808294 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc1f7b-7613-430f-9521-10751f3546d5 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:46.808565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc1f7b-7613-430f-9521-10751f3546d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:46.828489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-70dc1f7b-7613-430f-9521-10751f3546d5 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:46.829667 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-70dc1f7b-7613-430f-9521-10751f3546d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:46.909381 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 73/139] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:46.952303 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf310c-26af-4a6e-bd07-b9efdd884da8 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:47.078461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf310c-26af-4a6e-bd07-b9efdd884da8 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:47.118408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea022003-a0d2-41bb-b65b-7932d28644d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:47.119630 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea022003-a0d2-41bb-b65b-7932d28644d7 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:47.120118 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea022003-a0d2-41bb-b65b-7932d28644d7 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:47.120690 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea022003-a0d2-41bb-b65b-7932d28644d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:47.122100 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea022003-a0d2-41bb-b65b-7932d28644d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:47.123791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea022003-a0d2-41bb-b65b-7932d28644d7 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:47.125618 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 67/140] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:15:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:47.132253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc1f7b-7613-430f-9521-10751f3546d5 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:47.260625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc1f7b-7613-430f-9521-10751f3546d5 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:47.302970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7641c7-500b-4772-a785-bb797b065fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:47.303684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7641c7-500b-4772-a785-bb797b065fb9 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:47.303974 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7641c7-500b-4772-a785-bb797b065fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:47.304249 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7641c7-500b-4772-a785-bb797b065fb9 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:47.304961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7641c7-500b-4772-a785-bb797b065fb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:47.306396 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7641c7-500b-4772-a785-bb797b065fb9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:47.337746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 74/141] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:15:47 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:15:47.951415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bcf62-7190-498b-b219-80ee111080af None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:47.952446 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bcf62-7190-498b-b219-80ee111080af None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:47.952959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bcf62-7190-498b-b219-80ee111080af None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:47.954537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 68/142] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:47.961157 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a49a3b-597d-4969-b32e-6c22971aa19b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:47.961513 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a49a3b-597d-4969-b32e-6c22971aa19b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:47.961722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a49a3b-597d-4969-b32e-6c22971aa19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:47.989134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-14a49a3b-597d-4969-b32e-6c22971aa19b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:47.990365 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-14a49a3b-597d-4969-b32e-6c22971aa19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:48.075620 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 75/143] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:48.108199 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bcf62-7190-498b-b219-80ee111080af None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:48.238406 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bcf62-7190-498b-b219-80ee111080af None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:48.281071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876bfc6-ffe9-4396-9809-64f2304ccb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:48.282252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876bfc6-ffe9-4396-9809-64f2304ccb49 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:48.282748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876bfc6-ffe9-4396-9809-64f2304ccb49 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:48.283276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876bfc6-ffe9-4396-9809-64f2304ccb49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:48.294380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876bfc6-ffe9-4396-9809-64f2304ccb49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-0876bfc6-ffe9-4396-9809-64f2304ccb49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:48.302448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:48.303413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:48.304206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:48.304206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:48.304206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:48.304206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:48.304206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:48.304206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:48.304206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:48.304206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:48.304206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:48.304206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:48.304206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:15:48.304206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:48.306550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 69/144] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:48.312268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a49a3b-597d-4969-b32e-6c22971aa19b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:48.450921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a49a3b-597d-4969-b32e-6c22971aa19b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:48.490350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49509d30-122d-4c35-a3bf-bbebb474aa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:48.490966 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49509d30-122d-4c35-a3bf-bbebb474aa18 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:48.491275 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49509d30-122d-4c35-a3bf-bbebb474aa18 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:48.491557 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49509d30-122d-4c35-a3bf-bbebb474aa18 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:48.492310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49509d30-122d-4c35-a3bf-bbebb474aa18 None admin] RBAC: Adding query filter params (name=member) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:48.492617 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49509d30-122d-4c35-a3bf-bbebb474aa18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:48.493988 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49509d30-122d-4c35-a3bf-bbebb474aa18 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:48.503028 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 76/145] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:15:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:48.508308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876bfc6-ffe9-4396-9809-64f2304ccb49 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:48.622957 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c26e7-edc7-4b47-8e91-e7e63e39f9f2 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:48.623317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c26e7-edc7-4b47-8e91-e7e63e39f9f2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:48.623552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c26e7-edc7-4b47-8e91-e7e63e39f9f2 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:48.624920 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 77/146] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:48.631095 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26929ba5-ea3d-4730-a1c8-c4b79a4faf65 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:48.631280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26929ba5-ea3d-4730-a1c8-c4b79a4faf65 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:48.631490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26929ba5-ea3d-4730-a1c8-c4b79a4faf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:48.653478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876bfc6-ffe9-4396-9809-64f2304ccb49 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:48.663604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-26929ba5-ea3d-4730-a1c8-c4b79a4faf65 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:48.664797 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-26929ba5-ea3d-4730-a1c8-c4b79a4faf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:48.694805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941ab1b-0201-4823-9616-1ef9af36b817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:48.696092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941ab1b-0201-4823-9616-1ef9af36b817 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:48.696594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941ab1b-0201-4823-9616-1ef9af36b817 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:48.697097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941ab1b-0201-4823-9616-1ef9af36b817 None admin] PATH_INFO: `/v3/users/416a6d6800844d94a22ac939f6f6ba2a` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:48.703952 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941ab1b-0201-4823-9616-1ef9af36b817 None admin] RBAC: Authorizing `identity:get_user(user_id=416a6d6800844d94a22ac939f6f6ba2a)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:48.705676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941ab1b-0201-4823-9616-1ef9af36b817 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:48.708183 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 70/147] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:48 2022] GET /identity/v3/users/416a6d6800844d94a22ac939f6f6ba2a => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:48.713402 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941ab1b-0201-4823-9616-1ef9af36b817 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:48.769648 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 78/148] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:15:48.771460 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ce3ab-238f-4062-a8d7-5c7e3e5f28de None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:48.772180 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ce3ab-238f-4062-a8d7-5c7e3e5f28de None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:48.772180 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ce3ab-238f-4062-a8d7-5c7e3e5f28de None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:48.773629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 79/149] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:48.780455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7fb24-d37e-4bf9-a82c-505e1d7918b7 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:48.780741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7fb24-d37e-4bf9-a82c-505e1d7918b7 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:48.781020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7fb24-d37e-4bf9-a82c-505e1d7918b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:48.808615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-eac7fb24-d37e-4bf9-a82c-505e1d7918b7 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:48.811673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-eac7fb24-d37e-4bf9-a82c-505e1d7918b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:48.845987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941ab1b-0201-4823-9616-1ef9af36b817 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:48.915350 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6879e0-d3b9-49ed-abbb-0e0b392037b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:48.916025 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6879e0-d3b9-49ed-abbb-0e0b392037b5 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:48.916312 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6879e0-d3b9-49ed-abbb-0e0b392037b5 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:48.916593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6879e0-d3b9-49ed-abbb-0e0b392037b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:48.935944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 80/150] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:15:48.936880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7fb24-d37e-4bf9-a82c-505e1d7918b7 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:48.998767 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6879e0-d3b9-49ed-abbb-0e0b392037b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:49.000277 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6879e0-d3b9-49ed-abbb-0e0b392037b5 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:49.101555 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 71/151] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:15:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 18:15:49.102355 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6879e0-d3b9-49ed-abbb-0e0b392037b5 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:49.125067 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7fb24-d37e-4bf9-a82c-505e1d7918b7 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:49.192201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3112a-9206-4e66-ab57-41878dbafa0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:49.193006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3112a-9206-4e66-ab57-41878dbafa0d None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:49.193282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3112a-9206-4e66-ab57-41878dbafa0d None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:49.193549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3112a-9206-4e66-ab57-41878dbafa0d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:49.205158 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6879e0-d3b9-49ed-abbb-0e0b392037b5 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:49.218608 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3112a-9206-4e66-ab57-41878dbafa0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-84d3112a-9206-4e66-ab57-41878dbafa0d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:49.226375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:49.227301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:49.228056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:49.228056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:49.228056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:49.228056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:49.228056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:49.228056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:49.228056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:49.228056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:49.228056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:49.228056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:49.228056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:15:49.228056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:15:49.231891 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 81/152] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:49.233007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3112a-9206-4e66-ab57-41878dbafa0d None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:49.240554 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5552d-7281-4e8d-bbbc-4924aeb17168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:49.241380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5552d-7281-4e8d-bbbc-4924aeb17168 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:49.241671 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5552d-7281-4e8d-bbbc-4924aeb17168 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:49.241951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5552d-7281-4e8d-bbbc-4924aeb17168 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:49.253830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5552d-7281-4e8d-bbbc-4924aeb17168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-2aa5552d-7281-4e8d-bbbc-4924aeb17168 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:49.263418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:49.265549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:49.267758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:49.267758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:49.267758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:49.267758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:49.267758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:49.267758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:49.267758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:49.267758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:49.267758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:49.267758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:49.267758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:15:49.267758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:49.267758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 72/153] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:15:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:49.267758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5552d-7281-4e8d-bbbc-4924aeb17168 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:49.411833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5552d-7281-4e8d-bbbc-4924aeb17168 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:49.466838 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3112a-9206-4e66-ab57-41878dbafa0d None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:49.491676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f42a3e-f27b-498c-96fb-971616d8191f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:49.492364 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42a3e-f27b-498c-96fb-971616d8191f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:49.492652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42a3e-f27b-498c-96fb-971616d8191f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:49.492935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42a3e-f27b-498c-96fb-971616d8191f None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:49.493690 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f42a3e-f27b-498c-96fb-971616d8191f None admin] RBAC: Adding query filter params (name=reader) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:49.494003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f42a3e-f27b-498c-96fb-971616d8191f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:49.495407 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f42a3e-f27b-498c-96fb-971616d8191f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:49.514385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 73/154] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:15:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:49.515266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f42a3e-f27b-498c-96fb-971616d8191f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:49.542023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818155a9-3749-4f1d-bfa6-1fcd2affe223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:49.543016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818155a9-3749-4f1d-bfa6-1fcd2affe223 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:49.543425 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818155a9-3749-4f1d-bfa6-1fcd2affe223 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:49.543812 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818155a9-3749-4f1d-bfa6-1fcd2affe223 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:49.545745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818155a9-3749-4f1d-bfa6-1fcd2affe223 None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:49.547244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818155a9-3749-4f1d-bfa6-1fcd2affe223 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:49.549271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 82/155] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:49 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:49.550211 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818155a9-3749-4f1d-bfa6-1fcd2affe223 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:49.665638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818155a9-3749-4f1d-bfa6-1fcd2affe223 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:49.704323 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b61cd0-e6bb-4bf5-a302-b050692efd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:49.705199 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b61cd0-e6bb-4bf5-a302-b050692efd6d None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:49.705480 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b61cd0-e6bb-4bf5-a302-b050692efd6d None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:49.705756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b61cd0-e6bb-4bf5-a302-b050692efd6d None admin] PATH_INFO: `/v3/users/fda72ad9ae3049c08f96773911da779a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:49.707521 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b61cd0-e6bb-4bf5-a302-b050692efd6d None admin] RBAC: Authorizing `identity:get_user(user_id=fda72ad9ae3049c08f96773911da779a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:49.709374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b61cd0-e6bb-4bf5-a302-b050692efd6d None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:49.711391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 83/156] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:49 2022] GET /identity/v3/users/fda72ad9ae3049c08f96773911da779a => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:49.712155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b61cd0-e6bb-4bf5-a302-b050692efd6d None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:49.774471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f42a3e-f27b-498c-96fb-971616d8191f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:49.822567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b61cd0-e6bb-4bf5-a302-b050692efd6d None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:49.863579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38afc04-1383-4baf-848e-7a22a971f103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:49.864236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38afc04-1383-4baf-848e-7a22a971f103 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:49.864518 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38afc04-1383-4baf-848e-7a22a971f103 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:49.864792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38afc04-1383-4baf-848e-7a22a971f103 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:49.865616 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38afc04-1383-4baf-848e-7a22a971f103 None admin] RBAC: Adding query filter params (role.id=e1dddf0b88ee4578822ce51a2aba6760, scope.project.id=36617c1516b14f15a3b4a35692ff654f, user.id=416a6d6800844d94a22ac939f6f6ba2a) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:49.865921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38afc04-1383-4baf-848e-7a22a971f103 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:49.867379 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38afc04-1383-4baf-848e-7a22a971f103 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:49.873344 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612b509-7bf4-4fac-a20d-624ad6d680d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:49.874077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b509-7bf4-4fac-a20d-624ad6d680d3 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:49.874365 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b509-7bf4-4fac-a20d-624ad6d680d3 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:49.874644 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b509-7bf4-4fac-a20d-624ad6d680d3 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:49.875422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612b509-7bf4-4fac-a20d-624ad6d680d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:49.875732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612b509-7bf4-4fac-a20d-624ad6d680d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:49.877083 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612b509-7bf4-4fac-a20d-624ad6d680d3 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:49.884159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 84/157] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:15:49 2022] GET /identity/v3/role_assignments?user.id=416a6d6800844d94a22ac939f6f6ba2a&scope.project.id=36617c1516b14f15a3b4a35692ff654f&role.id=e1dddf0b88ee4578822ce51a2aba6760 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:49.889009 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204f558-a7a6-4c2d-9c48-1ef723c089fa None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:49.889311 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204f558-a7a6-4c2d-9c48-1ef723c089fa None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:49.889574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204f558-a7a6-4c2d-9c48-1ef723c089fa None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:49.892547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 85/158] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:49.893420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204f558-a7a6-4c2d-9c48-1ef723c089fa None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:49.896716 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 74/159] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:15:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:49.897863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac362f-5c4b-43a7-90fa-16aa194bfbe0 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:49.898156 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac362f-5c4b-43a7-90fa-16aa194bfbe0 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:49.898437 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac362f-5c4b-43a7-90fa-16aa194bfbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:49.940417 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-79ac362f-5c4b-43a7-90fa-16aa194bfbe0 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:49.941987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-79ac362f-5c4b-43a7-90fa-16aa194bfbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:50.027106 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204f558-a7a6-4c2d-9c48-1ef723c089fa None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:50.072892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62833d71-1ad2-4100-83e3-ca824053ac30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:50.073828 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62833d71-1ad2-4100-83e3-ca824053ac30 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:50.074110 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62833d71-1ad2-4100-83e3-ca824053ac30 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:50.074428 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62833d71-1ad2-4100-83e3-ca824053ac30 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:50.080033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62833d71-1ad2-4100-83e3-ca824053ac30 None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:50.081557 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62833d71-1ad2-4100-83e3-ca824053ac30 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:50.083992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 86/160] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:49 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:50.084817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62833d71-1ad2-4100-83e3-ca824053ac30 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:50.106349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 75/161] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:50.107764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6870bbf-3891-493f-bf8f-d3e2b0f7f87d None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:50.108063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6870bbf-3891-493f-bf8f-d3e2b0f7f87d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:50.108349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6870bbf-3891-493f-bf8f-d3e2b0f7f87d None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:50.112082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 76/162] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:50.112841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6870bbf-3891-493f-bf8f-d3e2b0f7f87d None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:50.282107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6870bbf-3891-493f-bf8f-d3e2b0f7f87d None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:50.305341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62833d71-1ad2-4100-83e3-ca824053ac30 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:50.318531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa97a5-76e3-411e-99f3-4303510e088f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:50.319502 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa97a5-76e3-411e-99f3-4303510e088f None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:50.319787 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa97a5-76e3-411e-99f3-4303510e088f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:50.320092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa97a5-76e3-411e-99f3-4303510e088f None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f/users/fda72ad9ae3049c08f96773911da779a/roles/3c0125c8cabb462197565509a20404dc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:50.322026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa97a5-76e3-411e-99f3-4303510e088f None admin] RBAC: Authorizing `identity:create_grant(project_id=36617c1516b14f15a3b4a35692ff654f, user_id=fda72ad9ae3049c08f96773911da779a, role_id=3c0125c8cabb462197565509a20404dc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:50.324499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa97a5-76e3-411e-99f3-4303510e088f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:50.341001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 77/163] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:15:50 2022] PUT /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f/users/fda72ad9ae3049c08f96773911da779a/roles/3c0125c8cabb462197565509a20404dc => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:15:50.343270 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbbd4b-4057-42aa-b790-cf5b7127c813 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:50.343575 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbbd4b-4057-42aa-b790-cf5b7127c813 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:50.343855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbbd4b-4057-42aa-b790-cf5b7127c813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:50.364033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-b0fbbd4b-4057-42aa-b790-cf5b7127c813 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:50.365615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b0fbbd4b-4057-42aa-b790-cf5b7127c813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:50.373783 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423f41b-f5ea-4572-814d-7f5913f973cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:50.374690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423f41b-f5ea-4572-814d-7f5913f973cd None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:50.374977 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423f41b-f5ea-4572-814d-7f5913f973cd None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:50.375330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423f41b-f5ea-4572-814d-7f5913f973cd None admin] PATH_INFO: `/v3/users/416a6d6800844d94a22ac939f6f6ba2a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:50.378188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423f41b-f5ea-4572-814d-7f5913f973cd None admin] RBAC: Authorizing `identity:get_user(user_id=416a6d6800844d94a22ac939f6f6ba2a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:50.380212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423f41b-f5ea-4572-814d-7f5913f973cd None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:50.381953 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 87/164] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:50 2022] GET /identity/v3/users/416a6d6800844d94a22ac939f6f6ba2a => generated 275 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:50.387088 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423f41b-f5ea-4572-814d-7f5913f973cd None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:50.443113 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 78/165] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:50.444029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbbd4b-4057-42aa-b790-cf5b7127c813 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:50.558578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbbd4b-4057-42aa-b790-cf5b7127c813 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:50.608679 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423f41b-f5ea-4572-814d-7f5913f973cd None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:50.632835 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18834307-1881-4a02-9033-ec473d5682cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:50.633529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18834307-1881-4a02-9033-ec473d5682cf None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:50.633812 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18834307-1881-4a02-9033-ec473d5682cf None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:50.634083 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18834307-1881-4a02-9033-ec473d5682cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:50.653850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0bc10-4d45-49d9-9471-f9f07d85bb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:50.654672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0bc10-4d45-49d9-9471-f9f07d85bb52 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:50.654952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0bc10-4d45-49d9-9471-f9f07d85bb52 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:50.655247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0bc10-4d45-49d9-9471-f9f07d85bb52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:50.677451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0bc10-4d45-49d9-9471-f9f07d85bb52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-a0a0bc10-4d45-49d9-9471-f9f07d85bb52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:50.689491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:50.690465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:50.691253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:50.691253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:50.691253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:50.691253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:50.691253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:50.691253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:50.691253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:50.691253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:50.691253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:50.691253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:50.691253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:50.691253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:15:50.691253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 88/166] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:15:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:50.692608 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0bc10-4d45-49d9-9471-f9f07d85bb52 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:50.717329 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18834307-1881-4a02-9033-ec473d5682cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:50.718807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18834307-1881-4a02-9033-ec473d5682cf None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:50.820328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 79/167] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:15:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:15:50.821107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18834307-1881-4a02-9033-ec473d5682cf None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:50.826021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0bc10-4d45-49d9-9471-f9f07d85bb52 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:50.900954 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedef70-10f4-4964-888b-e3681af2c0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:50.901779 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedef70-10f4-4964-888b-e3681af2c0d2 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:50.902063 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedef70-10f4-4964-888b-e3681af2c0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:50.902338 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedef70-10f4-4964-888b-e3681af2c0d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:50.922466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18834307-1881-4a02-9033-ec473d5682cf None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:50.932099 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdedef70-10f4-4964-888b-e3681af2c0d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-fdedef70-10f4-4964-888b-e3681af2c0d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:50.939918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:50.941997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:50.943886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:50.943886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:50.943886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:50.943886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:50.943886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:50.943886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:50.943886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:50.943886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:50.943886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:50.943886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:50.943886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:15:50.943886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:15:50.943886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 89/168] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:50.952218 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedef70-10f4-4964-888b-e3681af2c0d2 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:50.964529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8e22c-c3ba-49fd-a2e0-c268183b1866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:50.969692 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8e22c-c3ba-49fd-a2e0-c268183b1866 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:50.969692 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8e22c-c3ba-49fd-a2e0-c268183b1866 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:50.969692 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8e22c-c3ba-49fd-a2e0-c268183b1866 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:50.969692 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8e22c-c3ba-49fd-a2e0-c268183b1866 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:50.969692 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8e22c-c3ba-49fd-a2e0-c268183b1866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:50.969692 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8e22c-c3ba-49fd-a2e0-c268183b1866 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:50.993267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 80/169] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:50.994140 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8e22c-c3ba-49fd-a2e0-c268183b1866 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:51.171574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedef70-10f4-4964-888b-e3681af2c0d2 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:51.224181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8e22c-c3ba-49fd-a2e0-c268183b1866 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:51.243373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9b74c-4970-41ce-89f3-a5ec3222e1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:51.244244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9b74c-4970-41ce-89f3-a5ec3222e1b8 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:51.244527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9b74c-4970-41ce-89f3-a5ec3222e1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:51.244801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9b74c-4970-41ce-89f3-a5ec3222e1b8 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:51.246920 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9b74c-4970-41ce-89f3-a5ec3222e1b8 None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:51.248401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9b74c-4970-41ce-89f3-a5ec3222e1b8 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:51.250088 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 90/170] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:50 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:51.250860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9b74c-4970-41ce-89f3-a5ec3222e1b8 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:51.303900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f87d03-64fa-4559-a4a7-52dac9fccd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:51.304995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f87d03-64fa-4559-a4a7-52dac9fccd7e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:51.305472 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f87d03-64fa-4559-a4a7-52dac9fccd7e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:51.305934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f87d03-64fa-4559-a4a7-52dac9fccd7e None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:51.306874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f87d03-64fa-4559-a4a7-52dac9fccd7e None admin] RBAC: Adding query filter params (name=member) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:51.307414 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f87d03-64fa-4559-a4a7-52dac9fccd7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:51.308976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f87d03-64fa-4559-a4a7-52dac9fccd7e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:51.320769 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 81/171] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:15:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:51.321878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f87d03-64fa-4559-a4a7-52dac9fccd7e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:51.430108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9b74c-4970-41ce-89f3-a5ec3222e1b8 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:51.481878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3921279-db8f-427f-8494-3c5105bbc256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:51.481878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3921279-db8f-427f-8494-3c5105bbc256 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:51.481878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3921279-db8f-427f-8494-3c5105bbc256 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:51.481878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3921279-db8f-427f-8494-3c5105bbc256 None admin] PATH_INFO: `/v3/users/a379c334e6cd446c80b18fda6ce9162e` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:51.483299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3921279-db8f-427f-8494-3c5105bbc256 None admin] RBAC: Authorizing `identity:get_user(user_id=a379c334e6cd446c80b18fda6ce9162e)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:51.484752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3921279-db8f-427f-8494-3c5105bbc256 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:51.491452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 91/172] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:51 2022] GET /identity/v3/users/a379c334e6cd446c80b18fda6ce9162e => generated 247 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:51.491452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3921279-db8f-427f-8494-3c5105bbc256 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:51.539078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f87d03-64fa-4559-a4a7-52dac9fccd7e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:51.617955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c54296-b4d7-4d25-b778-e738565ed948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:51.618633 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c54296-b4d7-4d25-b778-e738565ed948 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:51.618918 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c54296-b4d7-4d25-b778-e738565ed948 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:51.619213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c54296-b4d7-4d25-b778-e738565ed948 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:51.620075 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c54296-b4d7-4d25-b778-e738565ed948 None admin] RBAC: Adding query filter params (role.id=9d45ca30c3584deabd69f2cd196ae5cc, scope.project.id=36617c1516b14f15a3b4a35692ff654f, user.id=416a6d6800844d94a22ac939f6f6ba2a) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:51.620391 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c54296-b4d7-4d25-b778-e738565ed948 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:51.621891 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c54296-b4d7-4d25-b778-e738565ed948 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:51.635327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 82/173] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:15:51 2022] GET /identity/v3/role_assignments?user.id=416a6d6800844d94a22ac939f6f6ba2a&scope.project.id=36617c1516b14f15a3b4a35692ff654f&role.id=9d45ca30c3584deabd69f2cd196ae5cc => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:51.636365 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c54296-b4d7-4d25-b778-e738565ed948 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:51.711493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3921279-db8f-427f-8494-3c5105bbc256 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:51.767077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c54296-b4d7-4d25-b778-e738565ed948 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:51.802450 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b47f4a-f975-4e03-a2b8-0504017bc1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:51.803380 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b47f4a-f975-4e03-a2b8-0504017bc1e1 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:51.803667 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b47f4a-f975-4e03-a2b8-0504017bc1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:51.803946 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b47f4a-f975-4e03-a2b8-0504017bc1e1 None admin] PATH_INFO: `/v3/users/416a6d6800844d94a22ac939f6f6ba2a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:51.805562 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b47f4a-f975-4e03-a2b8-0504017bc1e1 None admin] RBAC: Authorizing `identity:get_user(user_id=416a6d6800844d94a22ac939f6f6ba2a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:51.807042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b47f4a-f975-4e03-a2b8-0504017bc1e1 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:51.809319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a2e65-c19c-4ac4-b8d2-39aa5717e3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:51.809982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a2e65-c19c-4ac4-b8d2-39aa5717e3fb None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:51.810265 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a2e65-c19c-4ac4-b8d2-39aa5717e3fb None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:51.810539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a2e65-c19c-4ac4-b8d2-39aa5717e3fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:51.826159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 92/174] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:51 2022] GET /identity/v3/users/416a6d6800844d94a22ac939f6f6ba2a => generated 275 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:51.841671 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b47f4a-f975-4e03-a2b8-0504017bc1e1 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:51.975941 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a2e65-c19c-4ac4-b8d2-39aa5717e3fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:51.977445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a2e65-c19c-4ac4-b8d2-39aa5717e3fb None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:52.058235 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b47f4a-f975-4e03-a2b8-0504017bc1e1 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:52.120862 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c3d00-6838-4d99-b89c-c14072a3de26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:52.121673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c3d00-6838-4d99-b89c-c14072a3de26 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:52.121955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c3d00-6838-4d99-b89c-c14072a3de26 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:52.122393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c3d00-6838-4d99-b89c-c14072a3de26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:52.198322 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c3d00-6838-4d99-b89c-c14072a3de26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:52.200056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c3d00-6838-4d99-b89c-c14072a3de26 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:52.213071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 83/175] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:15:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:15:52.214573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec85dc-90c8-4c29-9ba0-90ec38348700 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:52.214804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec85dc-90c8-4c29-9ba0-90ec38348700 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:52.215086 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec85dc-90c8-4c29-9ba0-90ec38348700 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:52.216521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 84/176] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:52.223567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf83c03-73bc-4063-a708-aecdbf71c193 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:52.223898 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf83c03-73bc-4063-a708-aecdbf71c193 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:52.224127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf83c03-73bc-4063-a708-aecdbf71c193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:52.256094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-abf83c03-73bc-4063-a708-aecdbf71c193 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:52.257299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-abf83c03-73bc-4063-a708-aecdbf71c193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:52.387148 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 85/177] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:52.388989 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf83c03-73bc-4063-a708-aecdbf71c193 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:52.401023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 93/178] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:15:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:15:52.418924 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c3d00-6838-4d99-b89c-c14072a3de26 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:52.529665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf83c03-73bc-4063-a708-aecdbf71c193 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:52.543938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c3d00-6838-4d99-b89c-c14072a3de26 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:52.563012 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ea68a-abf6-4f17-aaa3-d4eab02c8d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:52.563906 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ea68a-abf6-4f17-aaa3-d4eab02c8d7c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:52.564188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ea68a-abf6-4f17-aaa3-d4eab02c8d7c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:52.564556 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ea68a-abf6-4f17-aaa3-d4eab02c8d7c None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:52.565715 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ea68a-abf6-4f17-aaa3-d4eab02c8d7c None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:52.567273 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ea68a-abf6-4f17-aaa3-d4eab02c8d7c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:52.569054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 86/179] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:52 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:52.569637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ea68a-abf6-4f17-aaa3-d4eab02c8d7c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:52.581517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2838-6965-4844-8435-b76dd2fe0a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:52.582428 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2838-6965-4844-8435-b76dd2fe0a09 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:52.582710 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2838-6965-4844-8435-b76dd2fe0a09 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:52.582985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2838-6965-4844-8435-b76dd2fe0a09 None admin] PATH_INFO: `/v3/projects/b64c14b29bb542b0b809d69c7c52b0be` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:52.584191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d2838-6965-4844-8435-b76dd2fe0a09 None admin] RBAC: Authorizing `identity:get_project(project_id=b64c14b29bb542b0b809d69c7c52b0be)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:52.585625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d2838-6965-4844-8435-b76dd2fe0a09 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:52.587341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 94/180] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:52 2022] GET /identity/v3/projects/b64c14b29bb542b0b809d69c7c52b0be => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:15:52.588597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a6970-1afb-4d9c-af9d-56cd3e536868 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:52.588886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a6970-1afb-4d9c-af9d-56cd3e536868 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:52.589163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a6970-1afb-4d9c-af9d-56cd3e536868 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:52.594796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 95/181] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:52.594796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d9d9b-c2fe-42fe-a49b-1431f60cff7f None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:52.594796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d9d9b-c2fe-42fe-a49b-1431f60cff7f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:52.594796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d9d9b-c2fe-42fe-a49b-1431f60cff7f None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:52.594796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 96/182] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:52.594796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d9d9b-c2fe-42fe-a49b-1431f60cff7f None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:52.674817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ea68a-abf6-4f17-aaa3-d4eab02c8d7c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:52.693958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d9d9b-c2fe-42fe-a49b-1431f60cff7f None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:52.708281 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f230af-1cad-424a-90d0-b55eb1cbf22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:52.709076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f230af-1cad-424a-90d0-b55eb1cbf22a None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:52.709351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f230af-1cad-424a-90d0-b55eb1cbf22a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:52.709619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f230af-1cad-424a-90d0-b55eb1cbf22a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:52.719428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f230af-1cad-424a-90d0-b55eb1cbf22a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:52.726852 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0b79f-a8f6-4369-8b0d-058842dccbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:52.727532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0b79f-a8f6-4369-8b0d-058842dccbde None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-65f230af-1cad-424a-90d0-b55eb1cbf22a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:52.727748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:52.728594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:52.729382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:52.729382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:52.729382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:52.729382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:52.729382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:52.729382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:52.729382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:52.729382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:52.729382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:52.729382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:52.729382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:52.729382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:52.729969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0b79f-a8f6-4369-8b0d-058842dccbde None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:52.729969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0b79f-a8f6-4369-8b0d-058842dccbde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:52.730086 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 87/183] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:15:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:52.730086 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f230af-1cad-424a-90d0-b55eb1cbf22a None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:52.733820 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0b79f-a8f6-4369-8b0d-058842dccbde None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=36617c1516b14f15a3b4a35692ff654f, user.id=a379c334e6cd446c80b18fda6ce9162e) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:52.734213 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0b79f-a8f6-4369-8b0d-058842dccbde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:52.735697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0b79f-a8f6-4369-8b0d-058842dccbde None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:52.741870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 97/184] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:15:52 2022] GET /identity/v3/role_assignments?user.id=a379c334e6cd446c80b18fda6ce9162e&scope.project.id=36617c1516b14f15a3b4a35692ff654f&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:52.748802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f150b0-6b69-4b2b-a7a3-5ce58f250ed6 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:52.749115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f150b0-6b69-4b2b-a7a3-5ce58f250ed6 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:52.749358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f150b0-6b69-4b2b-a7a3-5ce58f250ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:52.770092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-92f150b0-6b69-4b2b-a7a3-5ce58f250ed6 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:52.771398 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-92f150b0-6b69-4b2b-a7a3-5ce58f250ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:52.845534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f230af-1cad-424a-90d0-b55eb1cbf22a None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:52.848436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 98/185] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:52.850024 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d6bcb-eb95-4bd2-9daa-3f7c18a6d808 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:52.850239 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d6bcb-eb95-4bd2-9daa-3f7c18a6d808 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:52.850486 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d6bcb-eb95-4bd2-9daa-3f7c18a6d808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:52.885022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e119763-6991-440b-9351-833b9ac76033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:52.886085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e119763-6991-440b-9351-833b9ac76033 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:52.886760 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e119763-6991-440b-9351-833b9ac76033 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:52.887293 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e119763-6991-440b-9351-833b9ac76033 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:52.887733 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c18d6bcb-eb95-4bd2-9daa-3f7c18a6d808 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:52.888796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e119763-6991-440b-9351-833b9ac76033 None admin] RBAC: Adding query filter params (role.id=e1dddf0b88ee4578822ce51a2aba6760, scope.project.id=b64c14b29bb542b0b809d69c7c52b0be, user.id=416a6d6800844d94a22ac939f6f6ba2a) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:52.888904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c18d6bcb-eb95-4bd2-9daa-3f7c18a6d808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:52.889444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e119763-6991-440b-9351-833b9ac76033 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:52.891343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e119763-6991-440b-9351-833b9ac76033 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:52.898877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 88/186] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:15:52 2022] GET /identity/v3/role_assignments?user.id=416a6d6800844d94a22ac939f6f6ba2a&scope.project.id=b64c14b29bb542b0b809d69c7c52b0be&role.id=e1dddf0b88ee4578822ce51a2aba6760 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:52.900114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e119763-6991-440b-9351-833b9ac76033 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:53.005541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 99/187] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:53.006490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d6bcb-eb95-4bd2-9daa-3f7c18a6d808 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:53.048343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e119763-6991-440b-9351-833b9ac76033 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:53.091566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9524c1-ff55-4048-b917-2a18ae88f38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:53.092670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9524c1-ff55-4048-b917-2a18ae88f38c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:53.093150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9524c1-ff55-4048-b917-2a18ae88f38c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:53.093620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9524c1-ff55-4048-b917-2a18ae88f38c None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:53.094821 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9524c1-ff55-4048-b917-2a18ae88f38c None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:53.095613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9524c1-ff55-4048-b917-2a18ae88f38c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:53.097298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9524c1-ff55-4048-b917-2a18ae88f38c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:53.107540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 89/188] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:53.108740 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9524c1-ff55-4048-b917-2a18ae88f38c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:53.141355 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d6bcb-eb95-4bd2-9daa-3f7c18a6d808 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:53.178120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b5679-5b46-4ba2-b232-9c4fe86ed7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:53.178982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b5679-5b46-4ba2-b232-9c4fe86ed7bb None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:53.179305 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b5679-5b46-4ba2-b232-9c4fe86ed7bb None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:53.179585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b5679-5b46-4ba2-b232-9c4fe86ed7bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:53.180764 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b5679-5b46-4ba2-b232-9c4fe86ed7bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:53.182233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b5679-5b46-4ba2-b232-9c4fe86ed7bb None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:53.184244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 100/189] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:15:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:53.185031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b5679-5b46-4ba2-b232-9c4fe86ed7bb None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:53.217479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9524c1-ff55-4048-b917-2a18ae88f38c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:53.253096 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d2220-900f-4e5d-ab93-e1dbf816ad5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:53.254445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d2220-900f-4e5d-ab93-e1dbf816ad5d None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:53.254729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d2220-900f-4e5d-ab93-e1dbf816ad5d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:53.255002 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d2220-900f-4e5d-ab93-e1dbf816ad5d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:53.264948 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d2220-900f-4e5d-ab93-e1dbf816ad5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-338d2220-900f-4e5d-ab93-e1dbf816ad5d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:53.273727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:53.274775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:53.275919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:53.275919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:53.275919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:53.275919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:53.275919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:53.275919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:53.275919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:53.275919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:53.275919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:53.275919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:53.275919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:15:53.275919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:53.275919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 90/190] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:15:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:53.275919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d2220-900f-4e5d-ab93-e1dbf816ad5d None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:53.288030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b5679-5b46-4ba2-b232-9c4fe86ed7bb None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:53.319878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee5f71-573c-49e7-9f71-4cdb55d4fdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:53.320747 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee5f71-573c-49e7-9f71-4cdb55d4fdaf None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:53.321026 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee5f71-573c-49e7-9f71-4cdb55d4fdaf None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:53.321297 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee5f71-573c-49e7-9f71-4cdb55d4fdaf None admin] PATH_INFO: `/v3/users/af77c4a44fbe4461b06e97d8a2caa51b` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:53.334969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee5f71-573c-49e7-9f71-4cdb55d4fdaf None admin] RBAC: Authorizing `identity:get_user(user_id=af77c4a44fbe4461b06e97d8a2caa51b)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:53.336443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee5f71-573c-49e7-9f71-4cdb55d4fdaf None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:53.338071 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 101/191] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:53 2022] GET /identity/v3/users/af77c4a44fbe4461b06e97d8a2caa51b => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:53.338825 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee5f71-573c-49e7-9f71-4cdb55d4fdaf None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:53.372919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d2220-900f-4e5d-ab93-e1dbf816ad5d None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:53.404557 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8387bf-0687-4d33-a712-b6c2cfcecc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:53.405218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8387bf-0687-4d33-a712-b6c2cfcecc3a None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:53.405497 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8387bf-0687-4d33-a712-b6c2cfcecc3a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:53.405777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8387bf-0687-4d33-a712-b6c2cfcecc3a None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:53.406469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8387bf-0687-4d33-a712-b6c2cfcecc3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:53.407911 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8387bf-0687-4d33-a712-b6c2cfcecc3a None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:53.430830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee5f71-573c-49e7-9f71-4cdb55d4fdaf None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:53.433044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 91/192] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:15:53 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:15:53.434327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803eec4c-b4f1-41a1-84c8-8dd0f9ebf53c None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:53.434617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803eec4c-b4f1-41a1-84c8-8dd0f9ebf53c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:53.435110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803eec4c-b4f1-41a1-84c8-8dd0f9ebf53c None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:53.436423 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 92/193] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:53.436960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803eec4c-b4f1-41a1-84c8-8dd0f9ebf53c None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:53.470887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec02a0-4cc2-4050-b8c8-359a7bd45d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:53.472017 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec02a0-4cc2-4050-b8c8-359a7bd45d4b None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:53.472248 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec02a0-4cc2-4050-b8c8-359a7bd45d4b None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:53.472558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec02a0-4cc2-4050-b8c8-359a7bd45d4b None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:53.473611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec02a0-4cc2-4050-b8c8-359a7bd45d4b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:53.474192 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec02a0-4cc2-4050-b8c8-359a7bd45d4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:53.475537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec02a0-4cc2-4050-b8c8-359a7bd45d4b None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:53.489751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 102/194] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:15:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:53.489751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecbf2a-0c53-4819-ab0e-48e78f0304a6 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:53.489751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecbf2a-0c53-4819-ab0e-48e78f0304a6 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:53.489751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecbf2a-0c53-4819-ab0e-48e78f0304a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:53.507805 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-30ecbf2a-0c53-4819-ab0e-48e78f0304a6 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:53.509012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-30ecbf2a-0c53-4819-ab0e-48e78f0304a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:53.562489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803eec4c-b4f1-41a1-84c8-8dd0f9ebf53c None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:53.591206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 103/195] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:53.591206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecbf2a-0c53-4819-ab0e-48e78f0304a6 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:53.611362 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974aec3d-06fa-479c-85e4-e72f96c5165c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:53.612021 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aec3d-06fa-479c-85e4-e72f96c5165c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:53.612305 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aec3d-06fa-479c-85e4-e72f96c5165c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:53.612580 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aec3d-06fa-479c-85e4-e72f96c5165c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:53.688131 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974aec3d-06fa-479c-85e4-e72f96c5165c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:53.689533 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974aec3d-06fa-479c-85e4-e72f96c5165c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:53.693506 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecbf2a-0c53-4819-ab0e-48e78f0304a6 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:53.736533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82274ee6-fd82-44f7-9506-a93a49631073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:53.738708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82274ee6-fd82-44f7-9506-a93a49631073 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:53.738996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82274ee6-fd82-44f7-9506-a93a49631073 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:53.739354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82274ee6-fd82-44f7-9506-a93a49631073 None admin] PATH_INFO: `/v3/users/af77c4a44fbe4461b06e97d8a2caa51b` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:53.740613 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82274ee6-fd82-44f7-9506-a93a49631073 None admin] RBAC: Authorizing `identity:get_user(user_id=af77c4a44fbe4461b06e97d8a2caa51b)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:53.742071 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82274ee6-fd82-44f7-9506-a93a49631073 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:53.744138 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 104/196] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:53 2022] GET /identity/v3/users/af77c4a44fbe4461b06e97d8a2caa51b => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:53.744986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82274ee6-fd82-44f7-9506-a93a49631073 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:53.814683 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 93/197] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:15:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:15:53.815480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974aec3d-06fa-479c-85e4-e72f96c5165c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:53.853329 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82274ee6-fd82-44f7-9506-a93a49631073 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:53.890375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dd08d-8b92-4a89-b1d4-d009dfe57a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:53.891211 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dd08d-8b92-4a89-b1d4-d009dfe57a27 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:53.891504 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dd08d-8b92-4a89-b1d4-d009dfe57a27 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:53.891779 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dd08d-8b92-4a89-b1d4-d009dfe57a27 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:53.902541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2dd08d-8b92-4a89-b1d4-d009dfe57a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-9f2dd08d-8b92-4a89-b1d4-d009dfe57a27 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:53.910871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:53.911790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:53.912583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:53.912583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:53.912583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:53.912583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:53.912583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:53.912583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:53.912583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:53.912583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:53.912583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:53.912583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:53.912583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:15:53.912583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:15:53.912583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 105/198] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:53.913227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dd08d-8b92-4a89-b1d4-d009dfe57a27 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:53.937963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974aec3d-06fa-479c-85e4-e72f96c5165c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:53.980065 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf0aea-47ae-45f4-bc54-1fda7e17751f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:53.980816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf0aea-47ae-45f4-bc54-1fda7e17751f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:53.981169 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf0aea-47ae-45f4-bc54-1fda7e17751f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:53.981516 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf0aea-47ae-45f4-bc54-1fda7e17751f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:54.026973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dd08d-8b92-4a89-b1d4-d009dfe57a27 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:54.049939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf0aea-47ae-45f4-bc54-1fda7e17751f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:54.052186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf0aea-47ae-45f4-bc54-1fda7e17751f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:54.067148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292860f0-59bc-41b8-86f8-abb50ea1053d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:54.068187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292860f0-59bc-41b8-86f8-abb50ea1053d None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:54.068539 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292860f0-59bc-41b8-86f8-abb50ea1053d None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:54.068892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292860f0-59bc-41b8-86f8-abb50ea1053d None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:54.070142 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292860f0-59bc-41b8-86f8-abb50ea1053d None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:54.071722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292860f0-59bc-41b8-86f8-abb50ea1053d None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:54.073502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 106/199] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:53 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:54.074353 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292860f0-59bc-41b8-86f8-abb50ea1053d None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:54.152648 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 94/200] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:15:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:15:54.153611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf0aea-47ae-45f4-bc54-1fda7e17751f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:54.185465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292860f0-59bc-41b8-86f8-abb50ea1053d None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:54.218411 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01c97d-251e-42e8-bb26-0ddda6376c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:54.219114 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01c97d-251e-42e8-bb26-0ddda6376c38 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:54.219508 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01c97d-251e-42e8-bb26-0ddda6376c38 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:54.219859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01c97d-251e-42e8-bb26-0ddda6376c38 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:54.220753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01c97d-251e-42e8-bb26-0ddda6376c38 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:54.221130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01c97d-251e-42e8-bb26-0ddda6376c38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:54.223158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01c97d-251e-42e8-bb26-0ddda6376c38 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:54.231423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 107/201] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:15:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:54.232415 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01c97d-251e-42e8-bb26-0ddda6376c38 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:54.252556 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf0aea-47ae-45f4-bc54-1fda7e17751f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:54.287088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280b319-1cf7-405e-b993-2e11130f5155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:54.287820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280b319-1cf7-405e-b993-2e11130f5155 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:54.288071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280b319-1cf7-405e-b993-2e11130f5155 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:54.288349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280b319-1cf7-405e-b993-2e11130f5155 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:54.289175 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280b319-1cf7-405e-b993-2e11130f5155 None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=82a98033ef0f45f887ea92601788b835, user.id=af77c4a44fbe4461b06e97d8a2caa51b) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:54.289486 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280b319-1cf7-405e-b993-2e11130f5155 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:54.290906 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280b319-1cf7-405e-b993-2e11130f5155 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:54.296348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 95/202] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:15:54 2022] GET /identity/v3/role_assignments?user.id=af77c4a44fbe4461b06e97d8a2caa51b&scope.project.id=82a98033ef0f45f887ea92601788b835&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:54.297115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280b319-1cf7-405e-b993-2e11130f5155 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:54.343235 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01c97d-251e-42e8-bb26-0ddda6376c38 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:54.379798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60954ec2-9848-40c2-ae12-9a65c832107c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:54.381106 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60954ec2-9848-40c2-ae12-9a65c832107c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:54.381557 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60954ec2-9848-40c2-ae12-9a65c832107c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:54.381998 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60954ec2-9848-40c2-ae12-9a65c832107c None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:54.383427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60954ec2-9848-40c2-ae12-9a65c832107c None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:54.385056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60954ec2-9848-40c2-ae12-9a65c832107c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:54.387013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 108/203] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:54 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:54.395371 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60954ec2-9848-40c2-ae12-9a65c832107c None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:54.407351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280b319-1cf7-405e-b993-2e11130f5155 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:54.446035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a577e0-4f33-4432-86c8-a9b30b24923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:54.446939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a577e0-4f33-4432-86c8-a9b30b24923f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:54.447246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a577e0-4f33-4432-86c8-a9b30b24923f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:54.447527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a577e0-4f33-4432-86c8-a9b30b24923f None admin] PATH_INFO: `/v3/users/fda72ad9ae3049c08f96773911da779a` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:54.448790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a577e0-4f33-4432-86c8-a9b30b24923f None admin] RBAC: Authorizing `identity:get_user(user_id=fda72ad9ae3049c08f96773911da779a)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:54.450310 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a577e0-4f33-4432-86c8-a9b30b24923f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:54.452096 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 96/204] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:54 2022] GET /identity/v3/users/fda72ad9ae3049c08f96773911da779a => generated 289 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:54.458234 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a577e0-4f33-4432-86c8-a9b30b24923f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:54.507861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60954ec2-9848-40c2-ae12-9a65c832107c None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:54.543326 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66474547-61eb-472a-956c-3a6243f060fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:54.544386 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66474547-61eb-472a-956c-3a6243f060fb None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:54.544862 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66474547-61eb-472a-956c-3a6243f060fb None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:54.545335 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66474547-61eb-472a-956c-3a6243f060fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:54.546359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66474547-61eb-472a-956c-3a6243f060fb None admin] RBAC: Adding query filter params (role.id=9d45ca30c3584deabd69f2cd196ae5cc, scope.project.id=82a98033ef0f45f887ea92601788b835, user.id=af77c4a44fbe4461b06e97d8a2caa51b) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:54.546865 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66474547-61eb-472a-956c-3a6243f060fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:54.548639 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66474547-61eb-472a-956c-3a6243f060fb None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:54.554997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 109/205] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:15:54 2022] GET /identity/v3/role_assignments?user.id=af77c4a44fbe4461b06e97d8a2caa51b&scope.project.id=82a98033ef0f45f887ea92601788b835&role.id=9d45ca30c3584deabd69f2cd196ae5cc => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:54.562759 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a577e0-4f33-4432-86c8-a9b30b24923f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:54.603489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49f802-8bb5-402c-9226-8b7e7a629e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:54.604532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49f802-8bb5-402c-9226-8b7e7a629e91 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:54.605032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49f802-8bb5-402c-9226-8b7e7a629e91 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:54.605494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49f802-8bb5-402c-9226-8b7e7a629e91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:54.682200 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49f802-8bb5-402c-9226-8b7e7a629e91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:54.684134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49f802-8bb5-402c-9226-8b7e7a629e91 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:54.799986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 97/206] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:15:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 18:15:54.805500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66474547-61eb-472a-956c-3a6243f060fb None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:54.933301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66474547-61eb-472a-956c-3a6243f060fb None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:54.969732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22751c-ac2a-4b2d-9794-b5c3b9f20659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:54.970611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22751c-ac2a-4b2d-9794-b5c3b9f20659 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:54.970889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22751c-ac2a-4b2d-9794-b5c3b9f20659 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:54.971160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22751c-ac2a-4b2d-9794-b5c3b9f20659 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:54.972474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22751c-ac2a-4b2d-9794-b5c3b9f20659 None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:54.973898 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22751c-ac2a-4b2d-9794-b5c3b9f20659 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:54.975605 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 110/207] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:54 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:54.981513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49f802-8bb5-402c-9226-8b7e7a629e91 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:55.080399 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49f802-8bb5-402c-9226-8b7e7a629e91 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:55.113966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c30135-d4c0-4335-8a5f-af0ab1ea9158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:55.114624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c30135-d4c0-4335-8a5f-af0ab1ea9158 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:55.114899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c30135-d4c0-4335-8a5f-af0ab1ea9158 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:55.115163 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c30135-d4c0-4335-8a5f-af0ab1ea9158 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:55.116010 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c30135-d4c0-4335-8a5f-af0ab1ea9158 None admin] RBAC: Adding query filter params (role.id=3c0125c8cabb462197565509a20404dc, scope.project.id=36617c1516b14f15a3b4a35692ff654f, user.id=fda72ad9ae3049c08f96773911da779a) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:55.116313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c30135-d4c0-4335-8a5f-af0ab1ea9158 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:55.117746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c30135-d4c0-4335-8a5f-af0ab1ea9158 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:55.123820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 98/208] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:15:54 2022] GET /identity/v3/role_assignments?user.id=fda72ad9ae3049c08f96773911da779a&scope.project.id=36617c1516b14f15a3b4a35692ff654f&role.id=3c0125c8cabb462197565509a20404dc => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:56.073153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be07d31-ceae-49de-bfca-3e3891129807 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:56.073662 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be07d31-ceae-49de-bfca-3e3891129807 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:56.073735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be07d31-ceae-49de-bfca-3e3891129807 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:56.075280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 111/209] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:56.080558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022b68e-301a-42b3-aaf3-3f917a86b3e8 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:56.080934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022b68e-301a-42b3-aaf3-3f917a86b3e8 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:56.081111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022b68e-301a-42b3-aaf3-3f917a86b3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:56.098352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-4022b68e-301a-42b3-aaf3-3f917a86b3e8 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:56.099566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-4022b68e-301a-42b3-aaf3-3f917a86b3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:56.160489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b28a03-0108-4772-88bb-7e3077278490 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:56.161255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b28a03-0108-4772-88bb-7e3077278490 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:56.162042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b28a03-0108-4772-88bb-7e3077278490 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:56.163862 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 112/210] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:15:56.171228 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 99/211] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:56.172659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a203e1-1348-46b7-bf9b-fdc573537fca None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:56.173073 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a203e1-1348-46b7-bf9b-fdc573537fca None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:56.173484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a203e1-1348-46b7-bf9b-fdc573537fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:56.189734 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-56a203e1-1348-46b7-bf9b-fdc573537fca None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:56.191654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-56a203e1-1348-46b7-bf9b-fdc573537fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:56.196322 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022b68e-301a-42b3-aaf3-3f917a86b3e8 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:56.275198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 113/212] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:56.293434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a203e1-1348-46b7-bf9b-fdc573537fca None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:56.305363 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022b68e-301a-42b3-aaf3-3f917a86b3e8 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:56.339686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bcd27-f8ed-47e9-95da-14f38b0aee4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:56.340481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bcd27-f8ed-47e9-95da-14f38b0aee4f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:56.340762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bcd27-f8ed-47e9-95da-14f38b0aee4f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:56.341037 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bcd27-f8ed-47e9-95da-14f38b0aee4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:56.351376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bcd27-f8ed-47e9-95da-14f38b0aee4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-ba2bcd27-f8ed-47e9-95da-14f38b0aee4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:56.359165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:56.360050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:56.360834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:56.360834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:56.360834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:56.360834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:56.360834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:56.360834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:56.360834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:56.360834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:56.360834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:56.360834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:56.360834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:15:56.360834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:56.360834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 100/213] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:15:56.366195 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bcd27-f8ed-47e9-95da-14f38b0aee4f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:56.397006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a203e1-1348-46b7-bf9b-fdc573537fca None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:56.430424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526b8a8-175e-44c0-ab9f-43d10a3b9e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:56.431353 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526b8a8-175e-44c0-ab9f-43d10a3b9e61 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:56.431731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526b8a8-175e-44c0-ab9f-43d10a3b9e61 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:56.432158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526b8a8-175e-44c0-ab9f-43d10a3b9e61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:56.433324 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526b8a8-175e-44c0-ab9f-43d10a3b9e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:56.434849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526b8a8-175e-44c0-ab9f-43d10a3b9e61 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:56.441152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 114/214] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:15:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:15:56.442580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526b8a8-175e-44c0-ab9f-43d10a3b9e61 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:56.489707 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bcd27-f8ed-47e9-95da-14f38b0aee4f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:56.535995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60894e15-34e3-4518-a853-55e043c0d175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:56.536946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60894e15-34e3-4518-a853-55e043c0d175 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:56.537420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60894e15-34e3-4518-a853-55e043c0d175 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:56.537865 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60894e15-34e3-4518-a853-55e043c0d175 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:56.538789 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60894e15-34e3-4518-a853-55e043c0d175 None admin] RBAC: Adding query filter params (name=member) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:56.539290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60894e15-34e3-4518-a853-55e043c0d175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:56.540836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60894e15-34e3-4518-a853-55e043c0d175 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:56.541591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526b8a8-175e-44c0-ab9f-43d10a3b9e61 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:56.549884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 101/215] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:15:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:15:56.556009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60894e15-34e3-4518-a853-55e043c0d175 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:56.577804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeea06-82cb-4f21-a206-20e3d936af41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:56.578630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeea06-82cb-4f21-a206-20e3d936af41 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:56.578967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeea06-82cb-4f21-a206-20e3d936af41 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:56.579377 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeea06-82cb-4f21-a206-20e3d936af41 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:56.580455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aeea06-82cb-4f21-a206-20e3d936af41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:56.581668 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aeea06-82cb-4f21-a206-20e3d936af41 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:56.605241 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 115/216] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:15:56 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:15:56.673641 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60894e15-34e3-4518-a853-55e043c0d175 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:56.715039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f634445-8f38-4da9-b882-6718e7dfe0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:56.716376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f634445-8f38-4da9-b882-6718e7dfe0f4 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:56.716853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f634445-8f38-4da9-b882-6718e7dfe0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:56.717320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f634445-8f38-4da9-b882-6718e7dfe0f4 None admin] PATH_INFO: `/v3/users/d0f2a68cc3904b3d8ab2bf3fd83b2b9a` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:56.737955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f634445-8f38-4da9-b882-6718e7dfe0f4 None admin] RBAC: Authorizing `identity:get_user(user_id=d0f2a68cc3904b3d8ab2bf3fd83b2b9a)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:56.740576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f634445-8f38-4da9-b882-6718e7dfe0f4 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:56.743125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 102/217] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:56 2022] GET /identity/v3/users/d0f2a68cc3904b3d8ab2bf3fd83b2b9a => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:56.749532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeea06-82cb-4f21-a206-20e3d936af41 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:56.855950 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeea06-82cb-4f21-a206-20e3d936af41 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:56.890289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fab491-9310-44a1-9ec0-f89255e130a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:56.890989 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fab491-9310-44a1-9ec0-f89255e130a5 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:56.891281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fab491-9310-44a1-9ec0-f89255e130a5 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:56.891556 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fab491-9310-44a1-9ec0-f89255e130a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:56.935141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25f726-203b-4c73-b571-777ad75d8b24 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:56.935527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25f726-203b-4c73-b571-777ad75d8b24 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:56.935809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25f726-203b-4c73-b571-777ad75d8b24 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:56.937164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 103/218] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:56.943904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b11b8-f79c-4d0f-9ef5-9955578f3d7b None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:56.944464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b11b8-f79c-4d0f-9ef5-9955578f3d7b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:56.944950 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b11b8-f79c-4d0f-9ef5-9955578f3d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:56.970178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-904b11b8-f79c-4d0f-9ef5-9955578f3d7b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:56.970346 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fab491-9310-44a1-9ec0-f89255e130a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:56.971842 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-904b11b8-f79c-4d0f-9ef5-9955578f3d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:56.971971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fab491-9310-44a1-9ec0-f89255e130a5 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:57.062187 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 104/219] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:15:57.084844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b11b8-f79c-4d0f-9ef5-9955578f3d7b None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:57.096716 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 116/220] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:15:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:15:57.099167 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f853c0a-50fe-41ad-91ca-b5f324adc655 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:57.099475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f853c0a-50fe-41ad-91ca-b5f324adc655 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:57.099754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f853c0a-50fe-41ad-91ca-b5f324adc655 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:57.101174 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 117/221] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:57.106659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2822f-6d26-419e-af43-47a3b869dfc0 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:57.106659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2822f-6d26-419e-af43-47a3b869dfc0 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:57.107068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2822f-6d26-419e-af43-47a3b869dfc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:57.126193 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-92c2822f-6d26-419e-af43-47a3b869dfc0 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:57.127514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-92c2822f-6d26-419e-af43-47a3b869dfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:57.190763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b11b8-f79c-4d0f-9ef5-9955578f3d7b None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:57.195007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 118/222] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:15:57.203197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2822f-6d26-419e-af43-47a3b869dfc0 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:57.247075 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9017b6-1afa-4ca0-98dc-89b12f059e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:57.248135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9017b6-1afa-4ca0-98dc-89b12f059e24 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:57.248560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9017b6-1afa-4ca0-98dc-89b12f059e24 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:57.248975 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9017b6-1afa-4ca0-98dc-89b12f059e24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:57.259432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9017b6-1afa-4ca0-98dc-89b12f059e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-ad9017b6-1afa-4ca0-98dc-89b12f059e24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:57.267313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:57.268168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:57.270573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:57.270573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:57.270573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:57.270573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:57.270573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:57.270573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:57.270573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:57.270573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:57.270573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:57.270573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:57.270573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:15:57.270573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:57.272605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 105/223] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:15:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:15:57.273567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9017b6-1afa-4ca0-98dc-89b12f059e24 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:57.303144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2822f-6d26-419e-af43-47a3b869dfc0 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:57.336963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc1dd4-0445-4072-bab0-3c352f2a59fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:57.338087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc1dd4-0445-4072-bab0-3c352f2a59fe None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:57.338509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc1dd4-0445-4072-bab0-3c352f2a59fe None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:57.338924 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc1dd4-0445-4072-bab0-3c352f2a59fe None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:57.340266 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc1dd4-0445-4072-bab0-3c352f2a59fe None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:57.341863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc1dd4-0445-4072-bab0-3c352f2a59fe None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:57.343730 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 119/224] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:57 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:57.344620 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc1dd4-0445-4072-bab0-3c352f2a59fe None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:57.405976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9017b6-1afa-4ca0-98dc-89b12f059e24 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:57.449880 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36a042-b42a-462a-92a7-2d095391621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:57.451526 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36a042-b42a-462a-92a7-2d095391621d None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:57.451952 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36a042-b42a-462a-92a7-2d095391621d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:57.452371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36a042-b42a-462a-92a7-2d095391621d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:57.453280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc1dd4-0445-4072-bab0-3c352f2a59fe None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:57.464899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36a042-b42a-462a-92a7-2d095391621d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-fe36a042-b42a-462a-92a7-2d095391621d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:57.473859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:57.474923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:57.475743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:57.475743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:57.475743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:57.475743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:57.475743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:57.475743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:57.475743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:57.475743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:57.475743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:57.475743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:57.475743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:15:57.475743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:57.478114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 106/225] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:15:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:57.479062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36a042-b42a-462a-92a7-2d095391621d None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:57.490131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4555bb-229e-4ccc-b11c-e68214d44a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:57.491607 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4555bb-229e-4ccc-b11c-e68214d44a8d None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:57.492066 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4555bb-229e-4ccc-b11c-e68214d44a8d None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:57.492513 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4555bb-229e-4ccc-b11c-e68214d44a8d None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:57.493792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4555bb-229e-4ccc-b11c-e68214d44a8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:57.494637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4555bb-229e-4ccc-b11c-e68214d44a8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:57.496537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4555bb-229e-4ccc-b11c-e68214d44a8d None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:57.507592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 120/226] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:57.509255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4555bb-229e-4ccc-b11c-e68214d44a8d None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:57.586463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36a042-b42a-462a-92a7-2d095391621d None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:57.624493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8ea32-7731-42dd-8231-45ae42648b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:57.625156 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8ea32-7731-42dd-8231-45ae42648b92 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:57.625430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8ea32-7731-42dd-8231-45ae42648b92 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:57.625699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8ea32-7731-42dd-8231-45ae42648b92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:57.626522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8ea32-7731-42dd-8231-45ae42648b92 None admin] RBAC: Adding query filter params (role.id=e1dddf0b88ee4578822ce51a2aba6760, scope.project.id=82a98033ef0f45f887ea92601788b835, user.id=d0f2a68cc3904b3d8ab2bf3fd83b2b9a) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:57.626826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8ea32-7731-42dd-8231-45ae42648b92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:57.629111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8ea32-7731-42dd-8231-45ae42648b92 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:57.635048 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 107/227] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:15:57 2022] GET /identity/v3/role_assignments?user.id=d0f2a68cc3904b3d8ab2bf3fd83b2b9a&scope.project.id=82a98033ef0f45f887ea92601788b835&role.id=e1dddf0b88ee4578822ce51a2aba6760 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:57.635868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8ea32-7731-42dd-8231-45ae42648b92 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:57.641584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4555bb-229e-4ccc-b11c-e68214d44a8d None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:57.687598 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6644a-c4bb-47d8-a47b-81be24e2474b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:57.688651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6644a-c4bb-47d8-a47b-81be24e2474b None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:57.689229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6644a-c4bb-47d8-a47b-81be24e2474b None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:57.689648 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6644a-c4bb-47d8-a47b-81be24e2474b None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:57.690851 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e6644a-c4bb-47d8-a47b-81be24e2474b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:57.692148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e6644a-c4bb-47d8-a47b-81be24e2474b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:57.693792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e6644a-c4bb-47d8-a47b-81be24e2474b None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:57.706233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 121/228] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:15:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:57.716261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6644a-c4bb-47d8-a47b-81be24e2474b None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:57.749080 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8ea32-7731-42dd-8231-45ae42648b92 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:57.785057 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbb719-e39e-4511-b691-127ecd9330f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:57.786151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbb719-e39e-4511-b691-127ecd9330f4 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:57.786527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbb719-e39e-4511-b691-127ecd9330f4 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:57.786895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbb719-e39e-4511-b691-127ecd9330f4 None admin] PATH_INFO: `/v3/users/af77c4a44fbe4461b06e97d8a2caa51b` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:57.788260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbb719-e39e-4511-b691-127ecd9330f4 None admin] RBAC: Authorizing `identity:get_user(user_id=af77c4a44fbe4461b06e97d8a2caa51b)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:57.789816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbb719-e39e-4511-b691-127ecd9330f4 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:57.791984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 108/229] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:57 2022] GET /identity/v3/users/af77c4a44fbe4461b06e97d8a2caa51b => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:57.797447 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbb719-e39e-4511-b691-127ecd9330f4 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:57.831817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6644a-c4bb-47d8-a47b-81be24e2474b None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:57.866147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f26573-6ffc-43b9-941c-6fc72cc5bf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:57.867362 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f26573-6ffc-43b9-941c-6fc72cc5bf35 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:57.867828 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f26573-6ffc-43b9-941c-6fc72cc5bf35 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:57.868272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f26573-6ffc-43b9-941c-6fc72cc5bf35 None admin] PATH_INFO: `/v3/users/af77c4a44fbe4461b06e97d8a2caa51b` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:57.869676 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f26573-6ffc-43b9-941c-6fc72cc5bf35 None admin] RBAC: Authorizing `identity:get_user(user_id=af77c4a44fbe4461b06e97d8a2caa51b)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:57.871306 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f26573-6ffc-43b9-941c-6fc72cc5bf35 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:57.873558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 122/230] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:57 2022] GET /identity/v3/users/af77c4a44fbe4461b06e97d8a2caa51b => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:57.880082 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f26573-6ffc-43b9-941c-6fc72cc5bf35 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:57.904153 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbb719-e39e-4511-b691-127ecd9330f4 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:57.939697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192d14e-601d-445f-879e-86276862c190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:57.940566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192d14e-601d-445f-879e-86276862c190 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:57.940949 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192d14e-601d-445f-879e-86276862c190 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:57.941927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192d14e-601d-445f-879e-86276862c190 None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:57.943618 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192d14e-601d-445f-879e-86276862c190 None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:57.944837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192d14e-601d-445f-879e-86276862c190 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:57.947245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 109/231] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:57 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:15:57.952891 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192d14e-601d-445f-879e-86276862c190 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:57.992077 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f26573-6ffc-43b9-941c-6fc72cc5bf35 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:58.025950 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1575543-7b95-443c-b342-e6050708a8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:58.027136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1575543-7b95-443c-b342-e6050708a8d9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:58.027615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1575543-7b95-443c-b342-e6050708a8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:58.028060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1575543-7b95-443c-b342-e6050708a8d9 None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:58.029390 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1575543-7b95-443c-b342-e6050708a8d9 None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:58.031014 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1575543-7b95-443c-b342-e6050708a8d9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:58.033456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 123/232] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:15:57 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:15:58.039579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1575543-7b95-443c-b342-e6050708a8d9 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:58.061974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192d14e-601d-445f-879e-86276862c190 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:58.104718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef67a9d-e7e1-40ff-88c8-96fcfff3c9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:58.105614 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef67a9d-e7e1-40ff-88c8-96fcfff3c9f5 None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:58.107583 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef67a9d-e7e1-40ff-88c8-96fcfff3c9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:58.107583 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef67a9d-e7e1-40ff-88c8-96fcfff3c9f5 None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835/users/af77c4a44fbe4461b06e97d8a2caa51b/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:58.110892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef67a9d-e7e1-40ff-88c8-96fcfff3c9f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=82a98033ef0f45f887ea92601788b835, user_id=af77c4a44fbe4461b06e97d8a2caa51b, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:58.112354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef67a9d-e7e1-40ff-88c8-96fcfff3c9f5 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:58.140133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 110/233] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:15:57 2022] PUT /identity/v3/projects/82a98033ef0f45f887ea92601788b835/users/af77c4a44fbe4461b06e97d8a2caa51b/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:15:58.175272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1575543-7b95-443c-b342-e6050708a8d9 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:58.221777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575163f9-65a8-411f-8e53-5ddb25163f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:58.222631 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575163f9-65a8-411f-8e53-5ddb25163f6e None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:58.222912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575163f9-65a8-411f-8e53-5ddb25163f6e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:58.223202 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575163f9-65a8-411f-8e53-5ddb25163f6e None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835/users/af77c4a44fbe4461b06e97d8a2caa51b/roles/9d45ca30c3584deabd69f2cd196ae5cc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:58.225441 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575163f9-65a8-411f-8e53-5ddb25163f6e None admin] RBAC: Authorizing `identity:create_grant(project_id=82a98033ef0f45f887ea92601788b835, user_id=af77c4a44fbe4461b06e97d8a2caa51b, role_id=9d45ca30c3584deabd69f2cd196ae5cc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:58.227010 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575163f9-65a8-411f-8e53-5ddb25163f6e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:58.246135 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 124/234] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:15:58 2022] PUT /identity/v3/projects/82a98033ef0f45f887ea92601788b835/users/af77c4a44fbe4461b06e97d8a2caa51b/roles/9d45ca30c3584deabd69f2cd196ae5cc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:15:59.297584 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558ddea-27e0-443b-8718-7eb85d2bd425 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:59.298124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558ddea-27e0-443b-8718-7eb85d2bd425 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:59.298124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558ddea-27e0-443b-8718-7eb85d2bd425 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:59.299834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 111/235] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:15:59.305206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bc493-4e36-4ce8-a018-cf1acf7df4ce None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:59.305626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bc493-4e36-4ce8-a018-cf1acf7df4ce None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:59.305830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bc493-4e36-4ce8-a018-cf1acf7df4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:59.322955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-ea3bc493-4e36-4ce8-a018-cf1acf7df4ce None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:59.324161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ea3bc493-4e36-4ce8-a018-cf1acf7df4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:59.412703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 125/236] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:15:59.435024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558ddea-27e0-443b-8718-7eb85d2bd425 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:59.446652 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca44e29-b069-4f5a-9c84-d6889d726ca7 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:59.446930 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca44e29-b069-4f5a-9c84-d6889d726ca7 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:59.447230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca44e29-b069-4f5a-9c84-d6889d726ca7 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:59.448741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 126/237] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:15:59.453506 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0788b-e53b-47d1-88bf-b30f14cd0ab1 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:59.453790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0788b-e53b-47d1-88bf-b30f14cd0ab1 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:59.454071 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0788b-e53b-47d1-88bf-b30f14cd0ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:59.470273 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9aa0788b-e53b-47d1-88bf-b30f14cd0ab1 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:15:59.471461 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9aa0788b-e53b-47d1-88bf-b30f14cd0ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:15:59.581505 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558ddea-27e0-443b-8718-7eb85d2bd425 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:59.583420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 127/238] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:15:59.616723 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f56136-9733-4d4c-8669-3d656e4e7533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:59.617865 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f56136-9733-4d4c-8669-3d656e4e7533 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:59.618347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f56136-9733-4d4c-8669-3d656e4e7533 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:59.618863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f56136-9733-4d4c-8669-3d656e4e7533 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:59.620147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0788b-e53b-47d1-88bf-b30f14cd0ab1 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:59.645852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f56136-9733-4d4c-8669-3d656e4e7533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-52f56136-9733-4d4c-8669-3d656e4e7533 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:15:59.653490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:15:59.654360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:15:59.655147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:15:59.655147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:15:59.655147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:15:59.655147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:15:59.655147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:15:59.655147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:15:59.655147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:15:59.655147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:15:59.655147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:15:59.655147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:15:59.655147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:15:59.655147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:15:59.655147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 112/239] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:15:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:15:59.662244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f56136-9733-4d4c-8669-3d656e4e7533 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:59.743039 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0788b-e53b-47d1-88bf-b30f14cd0ab1 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:59.791022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125740a1-490c-47ca-883f-812c7214e7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:59.791874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125740a1-490c-47ca-883f-812c7214e7b0 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:59.792155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125740a1-490c-47ca-883f-812c7214e7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:59.792441 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125740a1-490c-47ca-883f-812c7214e7b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:59.795871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125740a1-490c-47ca-883f-812c7214e7b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:59.797327 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125740a1-490c-47ca-883f-812c7214e7b0 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:59.799126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 128/240] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:15:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:15:59.807758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125740a1-490c-47ca-883f-812c7214e7b0 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:15:59.864398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f56136-9733-4d4c-8669-3d656e4e7533 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:15:59.932772 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2317c18-bbff-434e-9cbb-f86cd821e6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:15:59.933467 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2317c18-bbff-434e-9cbb-f86cd821e6e9 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:15:59.933765 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2317c18-bbff-434e-9cbb-f86cd821e6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:15:59.934027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2317c18-bbff-434e-9cbb-f86cd821e6e9 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:15:59.934776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2317c18-bbff-434e-9cbb-f86cd821e6e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:15:59.935083 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2317c18-bbff-434e-9cbb-f86cd821e6e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:15:59.936500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2317c18-bbff-434e-9cbb-f86cd821e6e9 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:15:59.954493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 113/241] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:15:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:15:59.962048 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2317c18-bbff-434e-9cbb-f86cd821e6e9 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:00.002021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125740a1-490c-47ca-883f-812c7214e7b0 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:00.077478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2317c18-bbff-434e-9cbb-f86cd821e6e9 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:00.087113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa7444-5476-4362-9c1b-ffd122adb18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:00.088106 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa7444-5476-4362-9c1b-ffd122adb18c None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:00.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa7444-5476-4362-9c1b-ffd122adb18c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:00.089058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa7444-5476-4362-9c1b-ffd122adb18c None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:00.089949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa7444-5476-4362-9c1b-ffd122adb18c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:00.099306 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa7444-5476-4362-9c1b-ffd122adb18c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:00.115050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655b0cd-c1d5-4fc0-992a-17767faf6511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:00.115965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655b0cd-c1d5-4fc0-992a-17767faf6511 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:00.116254 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655b0cd-c1d5-4fc0-992a-17767faf6511 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:00.116535 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655b0cd-c1d5-4fc0-992a-17767faf6511 None admin] PATH_INFO: `/v3/users/1ecd4ba85c284f68a5e6aa14d0ebb45a` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:00.134389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 129/242] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:15:59 2022] POST /identity/v3/users => generated 293 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:16:00.139985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a655b0cd-c1d5-4fc0-992a-17767faf6511 None admin] RBAC: Authorizing `identity:get_user(user_id=1ecd4ba85c284f68a5e6aa14d0ebb45a)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:00.141481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a655b0cd-c1d5-4fc0-992a-17767faf6511 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:00.143674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 114/243] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:15:59 2022] GET /identity/v3/users/1ecd4ba85c284f68a5e6aa14d0ebb45a => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:00.152417 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa7444-5476-4362-9c1b-ffd122adb18c None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:00.290530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa7444-5476-4362-9c1b-ffd122adb18c None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:00.341821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63faf5f3-3f6c-4123-a85f-20e959eaffea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:00.342600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faf5f3-3f6c-4123-a85f-20e959eaffea None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:00.342817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faf5f3-3f6c-4123-a85f-20e959eaffea None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:00.343100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faf5f3-3f6c-4123-a85f-20e959eaffea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:00.416823 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63faf5f3-3f6c-4123-a85f-20e959eaffea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:00.418194 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63faf5f3-3f6c-4123-a85f-20e959eaffea None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:00.496056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765853f3-5d00-4a7a-ba64-098475ba55fd None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:00.496552 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765853f3-5d00-4a7a-ba64-098475ba55fd None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:00.496624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765853f3-5d00-4a7a-ba64-098475ba55fd None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:00.498126 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 115/244] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:16:00.504292 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e944de0-6222-48c4-9b72-f6b315c5e392 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:00.504585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e944de0-6222-48c4-9b72-f6b315c5e392 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:00.504868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e944de0-6222-48c4-9b72-f6b315c5e392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:00.528780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 130/245] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:00.535824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63faf5f3-3f6c-4123-a85f-20e959eaffea None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:00.538117 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-3e944de0-6222-48c4-9b72-f6b315c5e392 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:00.539372 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-3e944de0-6222-48c4-9b72-f6b315c5e392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:00.631297 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 116/246] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:00.652830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e944de0-6222-48c4-9b72-f6b315c5e392 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:00.657549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63faf5f3-3f6c-4123-a85f-20e959eaffea None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:00.712844 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801637a-11bb-491b-81d6-249ec1e5c114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:00.713648 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801637a-11bb-491b-81d6-249ec1e5c114 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:00.713930 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801637a-11bb-491b-81d6-249ec1e5c114 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:00.714488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801637a-11bb-491b-81d6-249ec1e5c114 None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:00.715434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801637a-11bb-491b-81d6-249ec1e5c114 None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:00.716903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801637a-11bb-491b-81d6-249ec1e5c114 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:00.723205 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 131/247] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:00 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:00.726014 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801637a-11bb-491b-81d6-249ec1e5c114 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:00.796379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e944de0-6222-48c4-9b72-f6b315c5e392 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:00.834134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fca22-0189-46b0-9bf1-fb9b1e2c8f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:00.835392 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fca22-0189-46b0-9bf1-fb9b1e2c8f4e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:00.835904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fca22-0189-46b0-9bf1-fb9b1e2c8f4e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:00.836377 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fca22-0189-46b0-9bf1-fb9b1e2c8f4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:00.846585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801637a-11bb-491b-81d6-249ec1e5c114 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:00.848000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52fca22-0189-46b0-9bf1-fb9b1e2c8f4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-b52fca22-0189-46b0-9bf1-fb9b1e2c8f4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:00.857130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:00.858031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:00.858830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:00.858830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:00.858830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:00.858830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:00.858830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:00.858830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:00.858830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:00.858830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:00.858830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:00.858830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:00.858830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:00.858830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:00.866556 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 117/248] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:00.868089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fca22-0189-46b0-9bf1-fb9b1e2c8f4e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:00.884741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd3bea-0664-4b58-9777-794f652a6481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:00.885425 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd3bea-0664-4b58-9777-794f652a6481 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:00.885771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd3bea-0664-4b58-9777-794f652a6481 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:00.886051 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd3bea-0664-4b58-9777-794f652a6481 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:00.886980 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd3bea-0664-4b58-9777-794f652a6481 None admin] RBAC: Adding query filter params (role.id=3c0125c8cabb462197565509a20404dc, scope.project.id=82a98033ef0f45f887ea92601788b835, user.id=1ecd4ba85c284f68a5e6aa14d0ebb45a) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:00.887295 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd3bea-0664-4b58-9777-794f652a6481 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:00.888903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd3bea-0664-4b58-9777-794f652a6481 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:00.895393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 132/249] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:00 2022] GET /identity/v3/role_assignments?user.id=1ecd4ba85c284f68a5e6aa14d0ebb45a&scope.project.id=82a98033ef0f45f887ea92601788b835&role.id=3c0125c8cabb462197565509a20404dc => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:00.999220 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fca22-0189-46b0-9bf1-fb9b1e2c8f4e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:01.043256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc611393-bfb5-4e49-9ed2-fe8f8721f506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:01.043958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc611393-bfb5-4e49-9ed2-fe8f8721f506 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:01.044241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc611393-bfb5-4e49-9ed2-fe8f8721f506 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:01.044518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc611393-bfb5-4e49-9ed2-fe8f8721f506 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:01.045282 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc611393-bfb5-4e49-9ed2-fe8f8721f506 None admin] RBAC: Adding query filter params (name=member) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:01.045590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc611393-bfb5-4e49-9ed2-fe8f8721f506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:01.046987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc611393-bfb5-4e49-9ed2-fe8f8721f506 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:01.059280 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 118/250] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:01.065311 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd3bea-0664-4b58-9777-794f652a6481 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:01.108902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231cd25-64e6-4629-82a9-56c4bfad7c59 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:01.109218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231cd25-64e6-4629-82a9-56c4bfad7c59 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:01.109474 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231cd25-64e6-4629-82a9-56c4bfad7c59 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:01.111199 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 119/251] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:01.124973 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c8df2-73b3-4738-921f-dda844bbbc14 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:01.125272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c8df2-73b3-4738-921f-dda844bbbc14 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:01.125546 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c8df2-73b3-4738-921f-dda844bbbc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:01.156308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-a48c8df2-73b3-4738-921f-dda844bbbc14 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:01.157582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a48c8df2-73b3-4738-921f-dda844bbbc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:01.191514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd3bea-0664-4b58-9777-794f652a6481 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:01.230332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b3a1d-ee8a-4f54-bb97-3f18796548e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:01.231231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b3a1d-ee8a-4f54-bb97-3f18796548e3 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:01.231517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b3a1d-ee8a-4f54-bb97-3f18796548e3 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:01.231793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b3a1d-ee8a-4f54-bb97-3f18796548e3 None admin] PATH_INFO: `/v3/users/d0f2a68cc3904b3d8ab2bf3fd83b2b9a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:01.232992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b3a1d-ee8a-4f54-bb97-3f18796548e3 None admin] RBAC: Authorizing `identity:get_user(user_id=d0f2a68cc3904b3d8ab2bf3fd83b2b9a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:01.234417 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b3a1d-ee8a-4f54-bb97-3f18796548e3 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:01.236402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 133/252] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:16:01 2022] GET /identity/v3/users/d0f2a68cc3904b3d8ab2bf3fd83b2b9a => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:01.241889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b3a1d-ee8a-4f54-bb97-3f18796548e3 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:01.309877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 120/253] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:01.353659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b3a1d-ee8a-4f54-bb97-3f18796548e3 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:01.361015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c8df2-73b3-4738-921f-dda844bbbc14 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:01.390004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacc56b-a6f8-44b8-8c60-cdc202b04a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:01.390870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacc56b-a6f8-44b8-8c60-cdc202b04a7a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:01.391146 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacc56b-a6f8-44b8-8c60-cdc202b04a7a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:01.391441 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacc56b-a6f8-44b8-8c60-cdc202b04a7a None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:01.393015 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacc56b-a6f8-44b8-8c60-cdc202b04a7a None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:01.395166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacc56b-a6f8-44b8-8c60-cdc202b04a7a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:01.397482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 134/254] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:01 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:01.403431 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacc56b-a6f8-44b8-8c60-cdc202b04a7a None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:01.526922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacc56b-a6f8-44b8-8c60-cdc202b04a7a None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:01.563163 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c8df2-73b3-4738-921f-dda844bbbc14 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:01.593357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76c4e2-1f23-4ab9-952f-752846803b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:01.594264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76c4e2-1f23-4ab9-952f-752846803b66 None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:01.594538 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76c4e2-1f23-4ab9-952f-752846803b66 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:01.594808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76c4e2-1f23-4ab9-952f-752846803b66 None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835/users/d0f2a68cc3904b3d8ab2bf3fd83b2b9a/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:01.600467 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76c4e2-1f23-4ab9-952f-752846803b66 None admin] RBAC: Authorizing `identity:create_grant(project_id=82a98033ef0f45f887ea92601788b835, user_id=d0f2a68cc3904b3d8ab2bf3fd83b2b9a, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:01.601590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92087d0d-0f6e-4105-a23f-0a8003da7dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:01.601902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76c4e2-1f23-4ab9-952f-752846803b66 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:01.602461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92087d0d-0f6e-4105-a23f-0a8003da7dc3 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:01.602685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92087d0d-0f6e-4105-a23f-0a8003da7dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:01.602960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92087d0d-0f6e-4105-a23f-0a8003da7dc3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:01.613754 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92087d0d-0f6e-4105-a23f-0a8003da7dc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-92087d0d-0f6e-4105-a23f-0a8003da7dc3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:01.623339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:01.624381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:01.625181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:01.625181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:01.625181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:01.625181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:01.625181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:01.625181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:01.625181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:01.625181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:01.625181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:01.625181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:01.625181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:01.625181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:01.625181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 121/255] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:16:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:01.625839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe236f-6590-4c32-8a86-b5a40d683151 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:01.626703 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe236f-6590-4c32-8a86-b5a40d683151 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:01.626814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 135/256] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:16:01 2022] PUT /identity/v3/projects/82a98033ef0f45f887ea92601788b835/users/d0f2a68cc3904b3d8ab2bf3fd83b2b9a/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:16:01.626889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe236f-6590-4c32-8a86-b5a40d683151 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:01.628129 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 122/257] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:01.633042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76c4e2-1f23-4ab9-952f-752846803b66 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:01.634518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda58b4-04ea-4039-a98d-5c8674523361 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:01.634812 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda58b4-04ea-4039-a98d-5c8674523361 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:01.635093 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda58b4-04ea-4039-a98d-5c8674523361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:01.653411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-7dda58b4-04ea-4039-a98d-5c8674523361 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:01.654610 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7dda58b4-04ea-4039-a98d-5c8674523361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:01.762123 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76c4e2-1f23-4ab9-952f-752846803b66 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:01.780918 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 123/258] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:01.798463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26572d-827a-4694-95bb-53970667c658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:01.799262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26572d-827a-4694-95bb-53970667c658 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:01.799542 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26572d-827a-4694-95bb-53970667c658 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:01.799810 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26572d-827a-4694-95bb-53970667c658 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:01.801133 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26572d-827a-4694-95bb-53970667c658 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:01.801433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26572d-827a-4694-95bb-53970667c658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:01.802866 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26572d-827a-4694-95bb-53970667c658 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:01.812090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 136/259] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:01.816868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda58b4-04ea-4039-a98d-5c8674523361 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:01.819770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26572d-827a-4694-95bb-53970667c658 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:01.957572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda58b4-04ea-4039-a98d-5c8674523361 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:02.014962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26572d-827a-4694-95bb-53970667c658 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:02.025642 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27785d31-3b54-4717-b285-41d622928263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:02.026453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27785d31-3b54-4717-b285-41d622928263 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:02.026732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27785d31-3b54-4717-b285-41d622928263 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:02.027006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27785d31-3b54-4717-b285-41d622928263 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:02.045584 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27785d31-3b54-4717-b285-41d622928263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-27785d31-3b54-4717-b285-41d622928263 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:02.056725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:02.057594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:02.058378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:02.058378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:02.058378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:02.058378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:02.058378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:02.058378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:02.058378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:02.058378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:02.058378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:02.058378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:02.058378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:16:02.058378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:02.058378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 124/260] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:16:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:02.068047 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27785d31-3b54-4717-b285-41d622928263 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:02.085930 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6415811-37cd-45cc-a1fc-8c427a739dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:02.086812 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6415811-37cd-45cc-a1fc-8c427a739dd4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:02.087094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6415811-37cd-45cc-a1fc-8c427a739dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:02.087394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6415811-37cd-45cc-a1fc-8c427a739dd4 None admin] PATH_INFO: `/v3/users/af77c4a44fbe4461b06e97d8a2caa51b` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:02.090103 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6415811-37cd-45cc-a1fc-8c427a739dd4 None admin] RBAC: Authorizing `identity:get_user(user_id=af77c4a44fbe4461b06e97d8a2caa51b)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:02.091588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6415811-37cd-45cc-a1fc-8c427a739dd4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:02.094668 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 137/261] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:16:01 2022] GET /identity/v3/users/af77c4a44fbe4461b06e97d8a2caa51b => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:02.109019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6415811-37cd-45cc-a1fc-8c427a739dd4 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:02.234361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27785d31-3b54-4717-b285-41d622928263 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:02.239412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6415811-37cd-45cc-a1fc-8c427a739dd4 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:02.272976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a515ede-85ec-461a-96df-917f64b6891f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:02.274500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a515ede-85ec-461a-96df-917f64b6891f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:02.275446 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a515ede-85ec-461a-96df-917f64b6891f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:02.276232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a515ede-85ec-461a-96df-917f64b6891f None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:02.277523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28943e5e-3d0e-440a-b542-542c85a19741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:02.278499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a515ede-85ec-461a-96df-917f64b6891f None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:02.279366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28943e5e-3d0e-440a-b542-542c85a19741 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:02.279894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a515ede-85ec-461a-96df-917f64b6891f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:02.281498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a515ede-85ec-461a-96df-917f64b6891f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:02.281981 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28943e5e-3d0e-440a-b542-542c85a19741 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:02.282453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28943e5e-3d0e-440a-b542-542c85a19741 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:02.303203 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 125/262] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:02.308736 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a515ede-85ec-461a-96df-917f64b6891f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:02.366399 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28943e5e-3d0e-440a-b542-542c85a19741 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:02.367892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28943e5e-3d0e-440a-b542-542c85a19741 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:02.464937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a515ede-85ec-461a-96df-917f64b6891f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:02.468569 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 138/263] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 18:16:02.476579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28943e5e-3d0e-440a-b542-542c85a19741 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:02.504240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8d9af-1afb-4639-995a-c4d605aa4751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:02.505128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8d9af-1afb-4639-995a-c4d605aa4751 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:02.505410 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8d9af-1afb-4639-995a-c4d605aa4751 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:02.505687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8d9af-1afb-4639-995a-c4d605aa4751 None admin] PATH_INFO: `/v3/users/af77c4a44fbe4461b06e97d8a2caa51b` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:02.506895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8d9af-1afb-4639-995a-c4d605aa4751 None admin] RBAC: Authorizing `identity:get_user(user_id=af77c4a44fbe4461b06e97d8a2caa51b)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:02.508361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8d9af-1afb-4639-995a-c4d605aa4751 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:02.510314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 126/264] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:16:02 2022] GET /identity/v3/users/af77c4a44fbe4461b06e97d8a2caa51b => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:02.515575 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8d9af-1afb-4639-995a-c4d605aa4751 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:02.645613 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28943e5e-3d0e-440a-b542-542c85a19741 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:02.689372 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8d9af-1afb-4639-995a-c4d605aa4751 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:02.704581 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496320d0-c5f1-400b-bead-b4e797c3716c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:02.705782 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496320d0-c5f1-400b-bead-b4e797c3716c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:02.706300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496320d0-c5f1-400b-bead-b4e797c3716c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:02.707124 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496320d0-c5f1-400b-bead-b4e797c3716c None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:02.708341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496320d0-c5f1-400b-bead-b4e797c3716c None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:02.709926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496320d0-c5f1-400b-bead-b4e797c3716c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:02.715199 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 139/265] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:02 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:02.718097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496320d0-c5f1-400b-bead-b4e797c3716c None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:02.751227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf97a7d-a8f1-4d88-8448-fbf6b292af94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:02.751890 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf97a7d-a8f1-4d88-8448-fbf6b292af94 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:02.752172 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf97a7d-a8f1-4d88-8448-fbf6b292af94 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:02.752448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf97a7d-a8f1-4d88-8448-fbf6b292af94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:02.876614 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf97a7d-a8f1-4d88-8448-fbf6b292af94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:02.878343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf97a7d-a8f1-4d88-8448-fbf6b292af94 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:02.896360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496320d0-c5f1-400b-bead-b4e797c3716c None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:02.965016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30110b-ef8b-4638-9ca9-1a3be5c4f798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:02.967294 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30110b-ef8b-4638-9ca9-1a3be5c4f798 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:02.967294 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30110b-ef8b-4638-9ca9-1a3be5c4f798 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:02.967294 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30110b-ef8b-4638-9ca9-1a3be5c4f798 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:02.967294 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30110b-ef8b-4638-9ca9-1a3be5c4f798 None admin] RBAC: Adding query filter params (role.id=9d45ca30c3584deabd69f2cd196ae5cc, scope.project.id=82a98033ef0f45f887ea92601788b835, user.id=af77c4a44fbe4461b06e97d8a2caa51b) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:02.967610 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30110b-ef8b-4638-9ca9-1a3be5c4f798 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:02.968775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30110b-ef8b-4638-9ca9-1a3be5c4f798 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:02.981217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 140/266] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:02 2022] GET /identity/v3/role_assignments?user.id=af77c4a44fbe4461b06e97d8a2caa51b&scope.project.id=82a98033ef0f45f887ea92601788b835&role.id=9d45ca30c3584deabd69f2cd196ae5cc => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:03.042935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 127/267] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 18:16:03.048267 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30110b-ef8b-4638-9ca9-1a3be5c4f798 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:03.171517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30110b-ef8b-4638-9ca9-1a3be5c4f798 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:03.213200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9300666-af43-4884-b268-4e9a9081d84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:03.214436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9300666-af43-4884-b268-4e9a9081d84b None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:03.214915 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9300666-af43-4884-b268-4e9a9081d84b None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:03.215411 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9300666-af43-4884-b268-4e9a9081d84b None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:03.216783 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9300666-af43-4884-b268-4e9a9081d84b None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:03.218402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9300666-af43-4884-b268-4e9a9081d84b None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:03.220656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 141/268] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:03 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:03.226292 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf97a7d-a8f1-4d88-8448-fbf6b292af94 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:03.361868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf97a7d-a8f1-4d88-8448-fbf6b292af94 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:03.400911 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef5576-8a6d-4d4d-baad-98e249e3ef7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:03.401593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef5576-8a6d-4d4d-baad-98e249e3ef7f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:03.401880 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef5576-8a6d-4d4d-baad-98e249e3ef7f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:03.402161 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef5576-8a6d-4d4d-baad-98e249e3ef7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:03.403008 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef5576-8a6d-4d4d-baad-98e249e3ef7f None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=82a98033ef0f45f887ea92601788b835, user.id=af77c4a44fbe4461b06e97d8a2caa51b) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:03.403383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef5576-8a6d-4d4d-baad-98e249e3ef7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:03.404855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef5576-8a6d-4d4d-baad-98e249e3ef7f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:03.416333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 128/269] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:03 2022] GET /identity/v3/role_assignments?user.id=af77c4a44fbe4461b06e97d8a2caa51b&scope.project.id=82a98033ef0f45f887ea92601788b835&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:03.609216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3ee25-37d6-425c-923e-5d83b3c1c1d2 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:03.610208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3ee25-37d6-425c-923e-5d83b3c1c1d2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:03.610693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3ee25-37d6-425c-923e-5d83b3c1c1d2 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:03.612542 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 142/270] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:03.618845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c37c34-c05d-47c8-888a-5467bc6e98a3 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:03.619135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c37c34-c05d-47c8-888a-5467bc6e98a3 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:03.619448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c37c34-c05d-47c8-888a-5467bc6e98a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:03.638974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-32c37c34-c05d-47c8-888a-5467bc6e98a3 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:03.640186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-32c37c34-c05d-47c8-888a-5467bc6e98a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:03.726508 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 129/271] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:03.769128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3ee25-37d6-425c-923e-5d83b3c1c1d2 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:03.882155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3ee25-37d6-425c-923e-5d83b3c1c1d2 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:03.923060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36884f0-4681-4972-9c17-728bfc7e064a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:03.923922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36884f0-4681-4972-9c17-728bfc7e064a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:03.924203 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36884f0-4681-4972-9c17-728bfc7e064a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:03.924476 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36884f0-4681-4972-9c17-728bfc7e064a None admin] PATH_INFO: `/v3/roles/member` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:03.937110 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36884f0-4681-4972-9c17-728bfc7e064a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-e36884f0-4681-4972-9c17-728bfc7e064a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:03.945162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:03.946221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:03.947000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:03.947000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:03.947000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:03.947000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:03.947000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:03.947000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:03.947000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:03.947000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:03.947000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:03.947000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:03.947000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:03.947000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:03.947662 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158dd75c-dd6a-4f04-ac2f-6321b71fcd6e None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:03.947662 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158dd75c-dd6a-4f04-ac2f-6321b71fcd6e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:03.947788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 143/272] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:03.947852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158dd75c-dd6a-4f04-ac2f-6321b71fcd6e None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:03.951368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 130/273] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:03.952025 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36884f0-4681-4972-9c17-728bfc7e064a None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:03.955908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d50c42-f88b-4847-a087-6a961533ae3d None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:03.956198 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d50c42-f88b-4847-a087-6a961533ae3d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:03.956480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d50c42-f88b-4847-a087-6a961533ae3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:03.987474 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-c7d50c42-f88b-4847-a087-6a961533ae3d None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:03.988748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c7d50c42-f88b-4847-a087-6a961533ae3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:04.072766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36884f0-4681-4972-9c17-728bfc7e064a None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:04.080833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 131/274] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:04.107475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d50c42-f88b-4847-a087-6a961533ae3d None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:04.123500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f078be-3146-417a-bf6d-44908bdc4b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:04.124498 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f078be-3146-417a-bf6d-44908bdc4b65 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:04.126953 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f078be-3146-417a-bf6d-44908bdc4b65 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:04.127526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f078be-3146-417a-bf6d-44908bdc4b65 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:04.129043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f078be-3146-417a-bf6d-44908bdc4b65 None admin] RBAC: Adding query filter params (name=member) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:04.129552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f078be-3146-417a-bf6d-44908bdc4b65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:04.131119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f078be-3146-417a-bf6d-44908bdc4b65 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:04.140450 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 144/275] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:04.152353 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f078be-3146-417a-bf6d-44908bdc4b65 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:04.237785 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d50c42-f88b-4847-a087-6a961533ae3d None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:04.266854 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f078be-3146-417a-bf6d-44908bdc4b65 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:04.280315 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69148742-528b-49b6-85ff-a2a5e2dfa664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:04.281770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69148742-528b-49b6-85ff-a2a5e2dfa664 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:04.282065 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69148742-528b-49b6-85ff-a2a5e2dfa664 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:04.282344 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69148742-528b-49b6-85ff-a2a5e2dfa664 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:04.283676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69148742-528b-49b6-85ff-a2a5e2dfa664 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:04.285961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69148742-528b-49b6-85ff-a2a5e2dfa664 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:04.292555 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 132/276] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:04.293703 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69148742-528b-49b6-85ff-a2a5e2dfa664 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:04.303973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c0c70-2c57-43a6-a315-65a05ff117f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:04.304942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c0c70-2c57-43a6-a315-65a05ff117f3 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:04.305289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c0c70-2c57-43a6-a315-65a05ff117f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:04.305628 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c0c70-2c57-43a6-a315-65a05ff117f3 None admin] PATH_INFO: `/v3/users/c35b5d90d66c4fcb9059468d42cf0e05` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:04.319858 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c0c70-2c57-43a6-a315-65a05ff117f3 None admin] RBAC: Authorizing `identity:get_user(user_id=c35b5d90d66c4fcb9059468d42cf0e05)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:04.321407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c0c70-2c57-43a6-a315-65a05ff117f3 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:04.328072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 145/277] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:16:04 2022] GET /identity/v3/users/c35b5d90d66c4fcb9059468d42cf0e05 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:04.329083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c0c70-2c57-43a6-a315-65a05ff117f3 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:04.393621 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69148742-528b-49b6-85ff-a2a5e2dfa664 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:04.426434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576a23b-ce66-4223-9891-04b873a07141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:04.427109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576a23b-ce66-4223-9891-04b873a07141 None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:04.427322 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c0c70-2c57-43a6-a315-65a05ff117f3 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:04.427458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576a23b-ce66-4223-9891-04b873a07141 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:04.427702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576a23b-ce66-4223-9891-04b873a07141 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:04.428414 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576a23b-ce66-4223-9891-04b873a07141 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:04.429841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576a23b-ce66-4223-9891-04b873a07141 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:04.450974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 133/278] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:16:04 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:16:04.452515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a42e3-8d57-464c-b26b-af33d13ac54a None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:04.452806 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a42e3-8d57-464c-b26b-af33d13ac54a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:04.453253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a42e3-8d57-464c-b26b-af33d13ac54a None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:04.454441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 134/279] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:04.460612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221b902-9966-4171-b414-d8d70995a89e None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:04.461170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221b902-9966-4171-b414-d8d70995a89e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:04.461591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221b902-9966-4171-b414-d8d70995a89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:04.474880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476a165-e4d0-4fd4-ae4c-dacdee1da5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:04.475887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476a165-e4d0-4fd4-ae4c-dacdee1da5e8 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:04.476265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476a165-e4d0-4fd4-ae4c-dacdee1da5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:04.476610 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476a165-e4d0-4fd4-ae4c-dacdee1da5e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:04.477572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476a165-e4d0-4fd4-ae4c-dacdee1da5e8 None admin] RBAC: Adding query filter params (role.id=e1dddf0b88ee4578822ce51a2aba6760, user.id=c35b5d90d66c4fcb9059468d42cf0e05, scope.system=all) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:04.477967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476a165-e4d0-4fd4-ae4c-dacdee1da5e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:04.479584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476a165-e4d0-4fd4-ae4c-dacdee1da5e8 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:04.481429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-e221b902-9966-4171-b414-d8d70995a89e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:04.482986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e221b902-9966-4171-b414-d8d70995a89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:04.485917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 146/280] 10.222.0.25 () {56 vars in 1308 bytes} [Mon May 9 18:16:04 2022] GET /identity/v3/role_assignments?user.id=c35b5d90d66c4fcb9059468d42cf0e05&scope.system=all&role.id=e1dddf0b88ee4578822ce51a2aba6760 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:04.570130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 135/281] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:04.594611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476a165-e4d0-4fd4-ae4c-dacdee1da5e8 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:04.712857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476a165-e4d0-4fd4-ae4c-dacdee1da5e8 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:04.746919 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701582e-db02-49f9-8c48-11c195d8bef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:04.748160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701582e-db02-49f9-8c48-11c195d8bef1 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:04.748644 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701582e-db02-49f9-8c48-11c195d8bef1 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:04.749113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701582e-db02-49f9-8c48-11c195d8bef1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:04.762021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701582e-db02-49f9-8c48-11c195d8bef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-7701582e-db02-49f9-8c48-11c195d8bef1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:04.770570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:04.771584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:04.772403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:04.772403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:04.772403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:04.772403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:04.772403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:04.772403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:04.772403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:04.772403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:04.772403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:04.772403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:04.772403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:16:04.772403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:04.774417 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 147/282] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:04.780611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221b902-9966-4171-b414-d8d70995a89e None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:04.901774 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221b902-9966-4171-b414-d8d70995a89e None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:04.942489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a278c8d-e970-4a4e-bd06-14ca745d32ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:04.943428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a278c8d-e970-4a4e-bd06-14ca745d32ba None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:04.943851 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a278c8d-e970-4a4e-bd06-14ca745d32ba None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:04.944266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a278c8d-e970-4a4e-bd06-14ca745d32ba None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:04.945173 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a278c8d-e970-4a4e-bd06-14ca745d32ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:04.945617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a278c8d-e970-4a4e-bd06-14ca745d32ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:04.947160 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a278c8d-e970-4a4e-bd06-14ca745d32ba None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:04.956061 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 136/283] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:04.959293 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffdceb-63fb-401a-ba77-401716cff740 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:04.959653 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffdceb-63fb-401a-ba77-401716cff740 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:04.960021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffdceb-63fb-401a-ba77-401716cff740 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:04.961450 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 148/284] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:04.965879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a278c8d-e970-4a4e-bd06-14ca745d32ba None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:04.969045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73e8d2-4c29-474a-854c-580513af4a4b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:04.969045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73e8d2-4c29-474a-854c-580513af4a4b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:04.969045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73e8d2-4c29-474a-854c-580513af4a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:04.995600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9f73e8d2-4c29-474a-854c-580513af4a4b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:04.997468 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9f73e8d2-4c29-474a-854c-580513af4a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:05.072970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 149/285] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:05.086419 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a278c8d-e970-4a4e-bd06-14ca745d32ba None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:05.096530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73e8d2-4c29-474a-854c-580513af4a4b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:05.126321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d3692-4034-4ed1-afef-d11855c38f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:05.127229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d3692-4034-4ed1-afef-d11855c38f1c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:05.127515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d3692-4034-4ed1-afef-d11855c38f1c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:05.128419 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d3692-4034-4ed1-afef-d11855c38f1c None admin] PATH_INFO: `/v3/users/1ecd4ba85c284f68a5e6aa14d0ebb45a` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:05.130365 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d3692-4034-4ed1-afef-d11855c38f1c None admin] RBAC: Authorizing `identity:get_user(user_id=1ecd4ba85c284f68a5e6aa14d0ebb45a)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:05.132400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d3692-4034-4ed1-afef-d11855c38f1c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:05.134342 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 137/286] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:16:04 2022] GET /identity/v3/users/1ecd4ba85c284f68a5e6aa14d0ebb45a => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:05.140817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d3692-4034-4ed1-afef-d11855c38f1c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:05.209880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73e8d2-4c29-474a-854c-580513af4a4b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:05.273724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae777c02-0165-4715-bfdf-63af9f11be9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:05.274525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae777c02-0165-4715-bfdf-63af9f11be9a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:05.274805 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae777c02-0165-4715-bfdf-63af9f11be9a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:05.275078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae777c02-0165-4715-bfdf-63af9f11be9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:05.288739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d3692-4034-4ed1-afef-d11855c38f1c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:05.295155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae777c02-0165-4715-bfdf-63af9f11be9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-ae777c02-0165-4715-bfdf-63af9f11be9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:05.309544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:05.310503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:05.311304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:05.311304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:05.311304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:05.311304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:05.311304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:05.311304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:05.311304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:05.311304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:05.311304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:05.311304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:05.311304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:05.311304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:05.316437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 150/287] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:05.318785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae777c02-0165-4715-bfdf-63af9f11be9a None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:05.364595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb46339-111b-4770-ab23-c53631973c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:05.365463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb46339-111b-4770-ab23-c53631973c4b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:05.365741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb46339-111b-4770-ab23-c53631973c4b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:05.366010 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb46339-111b-4770-ab23-c53631973c4b None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:05.367451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb46339-111b-4770-ab23-c53631973c4b None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:05.368879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb46339-111b-4770-ab23-c53631973c4b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:05.370564 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 138/288] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:05 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:05.376703 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb46339-111b-4770-ab23-c53631973c4b None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:05.505116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae777c02-0165-4715-bfdf-63af9f11be9a None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:05.554312 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb46339-111b-4770-ab23-c53631973c4b None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:05.561225 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b195a-a8e4-486f-bc76-415a7d057f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:05.561898 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b195a-a8e4-486f-bc76-415a7d057f82 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:05.562177 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b195a-a8e4-486f-bc76-415a7d057f82 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:05.562451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b195a-a8e4-486f-bc76-415a7d057f82 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:05.563232 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b195a-a8e4-486f-bc76-415a7d057f82 None admin] RBAC: Adding query filter params (name=member) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:05.563517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b195a-a8e4-486f-bc76-415a7d057f82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:05.564878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b195a-a8e4-486f-bc76-415a7d057f82 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:05.583612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 151/289] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:05.588768 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b195a-a8e4-486f-bc76-415a7d057f82 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:05.594577 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c1da5-b36b-430d-9bcd-b500f68b2b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:05.595872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c1da5-b36b-430d-9bcd-b500f68b2b3f None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:05.596358 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c1da5-b36b-430d-9bcd-b500f68b2b3f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:05.596833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c1da5-b36b-430d-9bcd-b500f68b2b3f None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835/users/1ecd4ba85c284f68a5e6aa14d0ebb45a/roles/3c0125c8cabb462197565509a20404dc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:05.599623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c1da5-b36b-430d-9bcd-b500f68b2b3f None admin] RBAC: Authorizing `identity:create_grant(project_id=82a98033ef0f45f887ea92601788b835, user_id=1ecd4ba85c284f68a5e6aa14d0ebb45a, role_id=3c0125c8cabb462197565509a20404dc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:05.601294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c1da5-b36b-430d-9bcd-b500f68b2b3f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:05.636273 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 139/290] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:16:05 2022] PUT /identity/v3/projects/82a98033ef0f45f887ea92601788b835/users/1ecd4ba85c284f68a5e6aa14d0ebb45a/roles/3c0125c8cabb462197565509a20404dc => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:16:05.697574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b195a-a8e4-486f-bc76-415a7d057f82 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:05.734984 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b71878-7cfb-4d9a-8f20-88758f5ce806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:05.735981 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b71878-7cfb-4d9a-8f20-88758f5ce806 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:05.736265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b71878-7cfb-4d9a-8f20-88758f5ce806 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:05.736541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b71878-7cfb-4d9a-8f20-88758f5ce806 None admin] PATH_INFO: `/v3/users/d0f2a68cc3904b3d8ab2bf3fd83b2b9a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:05.738012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b71878-7cfb-4d9a-8f20-88758f5ce806 None admin] RBAC: Authorizing `identity:get_user(user_id=d0f2a68cc3904b3d8ab2bf3fd83b2b9a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:05.739594 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b71878-7cfb-4d9a-8f20-88758f5ce806 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:05.741764 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 152/291] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:16:05 2022] GET /identity/v3/users/d0f2a68cc3904b3d8ab2bf3fd83b2b9a => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:05.747688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c1da5-b36b-430d-9bcd-b500f68b2b3f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:05.886528 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c1da5-b36b-430d-9bcd-b500f68b2b3f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:05.921679 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67aa84-01e0-4fbc-aeb5-fe1320c45fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:05.922346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67aa84-01e0-4fbc-aeb5-fe1320c45fa2 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:05.922627 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67aa84-01e0-4fbc-aeb5-fe1320c45fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:05.922902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67aa84-01e0-4fbc-aeb5-fe1320c45fa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:05.997791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67aa84-01e0-4fbc-aeb5-fe1320c45fa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:05.999280 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67aa84-01e0-4fbc-aeb5-fe1320c45fa2 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:06.097915 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 140/292] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:06.108236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b71878-7cfb-4d9a-8f20-88758f5ce806 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:06.258236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b71878-7cfb-4d9a-8f20-88758f5ce806 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:06.302579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22444af-0682-41b8-a686-c5a81b86f573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:06.303483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22444af-0682-41b8-a686-c5a81b86f573 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:06.303767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22444af-0682-41b8-a686-c5a81b86f573 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:06.304042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22444af-0682-41b8-a686-c5a81b86f573 None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:06.305212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22444af-0682-41b8-a686-c5a81b86f573 None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:06.306669 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22444af-0682-41b8-a686-c5a81b86f573 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:06.308454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 153/293] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:06 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:06.314546 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67aa84-01e0-4fbc-aeb5-fe1320c45fa2 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:06.428897 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67aa84-01e0-4fbc-aeb5-fe1320c45fa2 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:06.482241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966d23c-70ea-45de-8d69-b68072d5bcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:06.482911 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966d23c-70ea-45de-8d69-b68072d5bcc3 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:06.483204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966d23c-70ea-45de-8d69-b68072d5bcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:06.483490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966d23c-70ea-45de-8d69-b68072d5bcc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:06.484319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966d23c-70ea-45de-8d69-b68072d5bcc3 None admin] RBAC: Adding query filter params (role.id=e1dddf0b88ee4578822ce51a2aba6760, scope.project.id=82a98033ef0f45f887ea92601788b835, user.id=d0f2a68cc3904b3d8ab2bf3fd83b2b9a) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:06.484630 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966d23c-70ea-45de-8d69-b68072d5bcc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:06.486072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966d23c-70ea-45de-8d69-b68072d5bcc3 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:06.495874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 141/294] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:06 2022] GET /identity/v3/role_assignments?user.id=d0f2a68cc3904b3d8ab2bf3fd83b2b9a&scope.project.id=82a98033ef0f45f887ea92601788b835&role.id=e1dddf0b88ee4578822ce51a2aba6760 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:06.908931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29fc1d-14cb-4a75-b56c-3d5500303dc3 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:06.909450 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29fc1d-14cb-4a75-b56c-3d5500303dc3 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:06.909525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29fc1d-14cb-4a75-b56c-3d5500303dc3 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:06.910895 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 154/295] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:06.916316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739816cb-4be2-4fbd-90b4-453ec9de1671 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:06.916719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739816cb-4be2-4fbd-90b4-453ec9de1671 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:06.916900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739816cb-4be2-4fbd-90b4-453ec9de1671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:06.935580 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-739816cb-4be2-4fbd-90b4-453ec9de1671 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:06.936763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-739816cb-4be2-4fbd-90b4-453ec9de1671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:07.019263 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 142/296] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:07.035170 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29fc1d-14cb-4a75-b56c-3d5500303dc3 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:07.147932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29fc1d-14cb-4a75-b56c-3d5500303dc3 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:07.185764 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbf3cd-23f7-4a51-8c4a-bfab9d0ab7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:07.186496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbf3cd-23f7-4a51-8c4a-bfab9d0ab7ec None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:07.186780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbf3cd-23f7-4a51-8c4a-bfab9d0ab7ec None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:07.187060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbf3cd-23f7-4a51-8c4a-bfab9d0ab7ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:07.198813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbf3cd-23f7-4a51-8c4a-bfab9d0ab7ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-c2bbf3cd-23f7-4a51-8c4a-bfab9d0ab7ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:07.206775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:07.207703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:07.208491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:07.208491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:07.208491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:07.208491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:07.208491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:07.208491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:07.208491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:07.208491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:07.208491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:07.208491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:07.208491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:07.208491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:07.212890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 155/297] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:07.214301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739816cb-4be2-4fbd-90b4-453ec9de1671 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:07.313830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739816cb-4be2-4fbd-90b4-453ec9de1671 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:07.353486 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2fe74-68c6-45f8-9361-99f89052e0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:07.354224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2fe74-68c6-45f8-9361-99f89052e0ce None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:07.354509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2fe74-68c6-45f8-9361-99f89052e0ce None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:07.354788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2fe74-68c6-45f8-9361-99f89052e0ce None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:07.355627 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2fe74-68c6-45f8-9361-99f89052e0ce None admin] RBAC: Adding query filter params (name=member) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:07.355933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2fe74-68c6-45f8-9361-99f89052e0ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:07.357309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2fe74-68c6-45f8-9361-99f89052e0ce None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:07.371142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 143/298] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:07.372501 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbf3cd-23f7-4a51-8c4a-bfab9d0ab7ec None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:07.479852 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbf3cd-23f7-4a51-8c4a-bfab9d0ab7ec None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:07.513924 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69344631-926b-465d-aac3-2cc7c8d4de6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:07.514801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69344631-926b-465d-aac3-2cc7c8d4de6b None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:07.515074 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69344631-926b-465d-aac3-2cc7c8d4de6b None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:07.515371 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69344631-926b-465d-aac3-2cc7c8d4de6b None admin] PATH_INFO: `/v3/users/c35b5d90d66c4fcb9059468d42cf0e05` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:07.516542 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69344631-926b-465d-aac3-2cc7c8d4de6b None admin] RBAC: Authorizing `identity:get_user(user_id=c35b5d90d66c4fcb9059468d42cf0e05)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:07.517961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69344631-926b-465d-aac3-2cc7c8d4de6b None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:07.519922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 156/299] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:16:07 2022] GET /identity/v3/users/c35b5d90d66c4fcb9059468d42cf0e05 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:07.522951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e52ba-8309-4b15-a5b1-5e1552c370dd None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:07.523502 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e52ba-8309-4b15-a5b1-5e1552c370dd None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:07.523987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e52ba-8309-4b15-a5b1-5e1552c370dd None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:07.525797 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 144/300] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:07.528033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69344631-926b-465d-aac3-2cc7c8d4de6b None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:07.530826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152736f6-e099-4b57-93b6-655b560f380b None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:07.531116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152736f6-e099-4b57-93b6-655b560f380b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:07.531443 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152736f6-e099-4b57-93b6-655b560f380b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:07.548796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-152736f6-e099-4b57-93b6-655b560f380b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:07.549979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-152736f6-e099-4b57-93b6-655b560f380b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:07.623335 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 145/301] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:07.624694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b18ed-475b-4962-93f1-8e92ccaf7f3b None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:07.624978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b18ed-475b-4962-93f1-8e92ccaf7f3b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:07.625248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b18ed-475b-4962-93f1-8e92ccaf7f3b None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:07.626696 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 146/302] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:07.633302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67723e32-7c6e-43bd-8f52-b1202af592b0 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:07.634009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67723e32-7c6e-43bd-8f52-b1202af592b0 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:07.634824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67723e32-7c6e-43bd-8f52-b1202af592b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:07.643013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69344631-926b-465d-aac3-2cc7c8d4de6b None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:07.655215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-67723e32-7c6e-43bd-8f52-b1202af592b0 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:07.656371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-67723e32-7c6e-43bd-8f52-b1202af592b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:07.677136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226afd2e-7f37-450c-9feb-87eb7bf9ba17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:07.677997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226afd2e-7f37-450c-9feb-87eb7bf9ba17 None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:07.678276 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226afd2e-7f37-450c-9feb-87eb7bf9ba17 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:07.678551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226afd2e-7f37-450c-9feb-87eb7bf9ba17 None admin] PATH_INFO: `/v3/system/users/c35b5d90d66c4fcb9059468d42cf0e05/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:07.680045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226afd2e-7f37-450c-9feb-87eb7bf9ba17 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c35b5d90d66c4fcb9059468d42cf0e05, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:07.681424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226afd2e-7f37-450c-9feb-87eb7bf9ba17 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:07.697597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 157/303] 10.222.0.25 () {58 vars in 1334 bytes} [Mon May 9 18:16:07 2022] PUT /identity/v3/system/users/c35b5d90d66c4fcb9059468d42cf0e05/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:16:07.697597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226afd2e-7f37-450c-9feb-87eb7bf9ba17 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:07.742509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 147/304] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:07.767520 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67723e32-7c6e-43bd-8f52-b1202af592b0 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:07.827350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226afd2e-7f37-450c-9feb-87eb7bf9ba17 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:07.870568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40567b07-62fe-48df-8cc8-68261dabb8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:07.871410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40567b07-62fe-48df-8cc8-68261dabb8df None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:07.871696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40567b07-62fe-48df-8cc8-68261dabb8df None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:07.871974 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40567b07-62fe-48df-8cc8-68261dabb8df None admin] PATH_INFO: `/v3/domains/default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:07.873190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40567b07-62fe-48df-8cc8-68261dabb8df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:07.874804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40567b07-62fe-48df-8cc8-68261dabb8df None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:07.876659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 158/305] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:07.882908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40567b07-62fe-48df-8cc8-68261dabb8df None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:07.890271 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67723e32-7c6e-43bd-8f52-b1202af592b0 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:07.932336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7e16b-2854-4cca-a862-22d500b28814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:07.933920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7e16b-2854-4cca-a862-22d500b28814 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:07.934411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7e16b-2854-4cca-a862-22d500b28814 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:07.934919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7e16b-2854-4cca-a862-22d500b28814 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:07.945855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7e16b-2854-4cca-a862-22d500b28814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-d8f7e16b-2854-4cca-a862-22d500b28814 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:07.953687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:07.954749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:07.955749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:07.955749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:07.955749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:07.955749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:07.955749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:07.955749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:07.955749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:07.955749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:07.955749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:07.955749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:07.955749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:16:07.955749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:07.958608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 148/306] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:07.967237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7e16b-2854-4cca-a862-22d500b28814 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:08.023086 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40567b07-62fe-48df-8cc8-68261dabb8df None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:08.066730 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d31cb-62b7-4dd3-8b22-b01c9c7d47cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:08.067685 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d31cb-62b7-4dd3-8b22-b01c9c7d47cb None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:08.068132 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d31cb-62b7-4dd3-8b22-b01c9c7d47cb None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:08.068550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d31cb-62b7-4dd3-8b22-b01c9c7d47cb None admin] PATH_INFO: `/v3/groups` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:08.069848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d31cb-62b7-4dd3-8b22-b01c9c7d47cb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:08.071781 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d31cb-62b7-4dd3-8b22-b01c9c7d47cb None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:08.082512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7e16b-2854-4cca-a862-22d500b28814 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:08.093571 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 159/307] 10.222.0.25 () {60 vars in 1134 bytes} [Mon May 9 18:16:07 2022] POST /identity/v3/groups => generated 229 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:16:08.125518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306040d-6366-40ab-a4ec-103e7983d34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:08.126520 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306040d-6366-40ab-a4ec-103e7983d34c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:08.126950 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306040d-6366-40ab-a4ec-103e7983d34c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:08.127441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306040d-6366-40ab-a4ec-103e7983d34c None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:08.129352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306040d-6366-40ab-a4ec-103e7983d34c None admin] RBAC: Adding query filter params (name=reader) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:08.129830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306040d-6366-40ab-a4ec-103e7983d34c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:08.131479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306040d-6366-40ab-a4ec-103e7983d34c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:08.141578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 149/308] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:08.148423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d31cb-62b7-4dd3-8b22-b01c9c7d47cb None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:08.175070 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff5717-1a00-4efa-89a1-df8c5350ea7d None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:08.175468 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff5717-1a00-4efa-89a1-df8c5350ea7d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:08.175686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff5717-1a00-4efa-89a1-df8c5350ea7d None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:08.177329 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 150/309] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:16:08.182383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feae8a-3a20-4cef-b974-d1d54532c913 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:08.182668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feae8a-3a20-4cef-b974-d1d54532c913 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:08.182950 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feae8a-3a20-4cef-b974-d1d54532c913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:08.202033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-54feae8a-3a20-4cef-b974-d1d54532c913 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:08.203281 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-54feae8a-3a20-4cef-b974-d1d54532c913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:08.260973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d31cb-62b7-4dd3-8b22-b01c9c7d47cb None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:08.278839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 151/310] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:08.302786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feae8a-3a20-4cef-b974-d1d54532c913 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:08.303060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad2843-b524-4e70-b0d9-448cdbc44f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:08.303986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad2843-b524-4e70-b0d9-448cdbc44f3f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:08.304279 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad2843-b524-4e70-b0d9-448cdbc44f3f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:08.304549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad2843-b524-4e70-b0d9-448cdbc44f3f None admin] PATH_INFO: `/v3/users/a012a37135e443efb771b66b561863c4` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:08.321758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad2843-b524-4e70-b0d9-448cdbc44f3f None admin] RBAC: Authorizing `identity:get_user(user_id=a012a37135e443efb771b66b561863c4)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:08.323309 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad2843-b524-4e70-b0d9-448cdbc44f3f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:08.325379 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 160/311] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:16:08 2022] GET /identity/v3/users/a012a37135e443efb771b66b561863c4 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:08.330736 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad2843-b524-4e70-b0d9-448cdbc44f3f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:08.417537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feae8a-3a20-4cef-b974-d1d54532c913 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:08.452028 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a919954-e28a-48c8-9f42-5bfc3373293e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:08.453326 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a919954-e28a-48c8-9f42-5bfc3373293e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:08.453807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a919954-e28a-48c8-9f42-5bfc3373293e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:08.454289 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a919954-e28a-48c8-9f42-5bfc3373293e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:08.458164 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad2843-b524-4e70-b0d9-448cdbc44f3f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:08.464918 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a919954-e28a-48c8-9f42-5bfc3373293e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-5a919954-e28a-48c8-9f42-5bfc3373293e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:08.473017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:08.473883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:08.474653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:08.474653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:08.474653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:08.474653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:08.474653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:08.474653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:08.474653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:08.474653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:08.474653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:08.474653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:08.474653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:16:08.474653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:08.477330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 152/312] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:08.482575 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a919954-e28a-48c8-9f42-5bfc3373293e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:08.504992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bd0d4-785c-4882-a6c4-e435d8762012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:08.505663 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bd0d4-785c-4882-a6c4-e435d8762012 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:08.505948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bd0d4-785c-4882-a6c4-e435d8762012 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:08.506230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bd0d4-785c-4882-a6c4-e435d8762012 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:08.507058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bd0d4-785c-4882-a6c4-e435d8762012 None admin] RBAC: Adding query filter params (role.id=3c0125c8cabb462197565509a20404dc, user.id=a012a37135e443efb771b66b561863c4, scope.system=all) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:08.507846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bd0d4-785c-4882-a6c4-e435d8762012 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:08.509579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bd0d4-785c-4882-a6c4-e435d8762012 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:08.516690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 161/313] 10.222.0.25 () {56 vars in 1308 bytes} [Mon May 9 18:16:08 2022] GET /identity/v3/role_assignments?user.id=a012a37135e443efb771b66b561863c4&scope.system=all&role.id=3c0125c8cabb462197565509a20404dc => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:08.597112 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a919954-e28a-48c8-9f42-5bfc3373293e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:08.634267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5dc2f-196e-42ad-b328-f4d4e5ad5bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:08.634950 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5dc2f-196e-42ad-b328-f4d4e5ad5bd8 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:08.635332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5dc2f-196e-42ad-b328-f4d4e5ad5bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:08.635615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5dc2f-196e-42ad-b328-f4d4e5ad5bd8 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:08.636438 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5dc2f-196e-42ad-b328-f4d4e5ad5bd8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:08.636740 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5dc2f-196e-42ad-b328-f4d4e5ad5bd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:08.638178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5dc2f-196e-42ad-b328-f4d4e5ad5bd8 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:08.647481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 153/314] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:08.652950 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bd0d4-785c-4882-a6c4-e435d8762012 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:08.760176 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bd0d4-785c-4882-a6c4-e435d8762012 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:08.793240 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e8769-2761-452c-9d91-fcef6df76157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:08.794111 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e8769-2761-452c-9d91-fcef6df76157 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:08.794397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e8769-2761-452c-9d91-fcef6df76157 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:08.794672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e8769-2761-452c-9d91-fcef6df76157 None admin] PATH_INFO: `/v3/users/1ecd4ba85c284f68a5e6aa14d0ebb45a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:08.795874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e8769-2761-452c-9d91-fcef6df76157 None admin] RBAC: Authorizing `identity:get_user(user_id=1ecd4ba85c284f68a5e6aa14d0ebb45a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:08.797308 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e8769-2761-452c-9d91-fcef6df76157 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:08.799085 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 162/315] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:16:08 2022] GET /identity/v3/users/1ecd4ba85c284f68a5e6aa14d0ebb45a => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:08.804987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5dc2f-196e-42ad-b328-f4d4e5ad5bd8 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:08.909378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5dc2f-196e-42ad-b328-f4d4e5ad5bd8 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:08.943397 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c5264-198e-4c50-b7ef-011e41e75dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:08.944163 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c5264-198e-4c50-b7ef-011e41e75dd0 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:08.944321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c5264-198e-4c50-b7ef-011e41e75dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:08.944579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c5264-198e-4c50-b7ef-011e41e75dd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:09.012196 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c5264-198e-4c50-b7ef-011e41e75dd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:09.013646 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c5264-198e-4c50-b7ef-011e41e75dd0 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:09.110032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 154/316] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:09.115912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e8769-2761-452c-9d91-fcef6df76157 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:09.214783 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e8769-2761-452c-9d91-fcef6df76157 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:09.248552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a9874-fce5-4bc4-b6ce-4a1f3a157612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:09.249413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a9874-fce5-4bc4-b6ce-4a1f3a157612 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:09.249693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a9874-fce5-4bc4-b6ce-4a1f3a157612 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:09.249967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a9874-fce5-4bc4-b6ce-4a1f3a157612 None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:09.251141 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a9874-fce5-4bc4-b6ce-4a1f3a157612 None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:09.252598 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a9874-fce5-4bc4-b6ce-4a1f3a157612 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:09.254887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 163/317] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:09 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:09.260695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c5264-198e-4c50-b7ef-011e41e75dd0 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:09.364919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c5264-198e-4c50-b7ef-011e41e75dd0 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:09.399875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd5c3d-06bb-4aff-bc5f-5e6a9f46d0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:09.400536 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd5c3d-06bb-4aff-bc5f-5e6a9f46d0a7 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:09.400819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd5c3d-06bb-4aff-bc5f-5e6a9f46d0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:09.401097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd5c3d-06bb-4aff-bc5f-5e6a9f46d0a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:09.401933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd5c3d-06bb-4aff-bc5f-5e6a9f46d0a7 None admin] RBAC: Adding query filter params (role.id=3c0125c8cabb462197565509a20404dc, scope.project.id=82a98033ef0f45f887ea92601788b835, user.id=1ecd4ba85c284f68a5e6aa14d0ebb45a) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:09.402243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd5c3d-06bb-4aff-bc5f-5e6a9f46d0a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:09.403711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd5c3d-06bb-4aff-bc5f-5e6a9f46d0a7 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:09.410536 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 155/318] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:09 2022] GET /identity/v3/role_assignments?user.id=1ecd4ba85c284f68a5e6aa14d0ebb45a&scope.project.id=82a98033ef0f45f887ea92601788b835&role.id=3c0125c8cabb462197565509a20404dc => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:10.009628 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579ced0-501d-4f91-9a31-a8e6989950cc None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:10.009936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579ced0-501d-4f91-9a31-a8e6989950cc None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:10.010192 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579ced0-501d-4f91-9a31-a8e6989950cc None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:10.011726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 164/319] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:10.016831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ea26f-012e-48ef-b6e8-9409e15f4c25 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:10.017272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ea26f-012e-48ef-b6e8-9409e15f4c25 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:10.017429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ea26f-012e-48ef-b6e8-9409e15f4c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:10.033440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-9f6ea26f-012e-48ef-b6e8-9409e15f4c25 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:10.034640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9f6ea26f-012e-48ef-b6e8-9409e15f4c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:10.098090 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 156/320] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:10.121443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579ced0-501d-4f91-9a31-a8e6989950cc None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:10.217120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579ced0-501d-4f91-9a31-a8e6989950cc None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:10.250829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2f5fa-89da-4dde-a791-b215fe968d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:10.251701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2f5fa-89da-4dde-a791-b215fe968d56 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:10.251985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2f5fa-89da-4dde-a791-b215fe968d56 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:10.252261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2f5fa-89da-4dde-a791-b215fe968d56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:10.265057 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2f5fa-89da-4dde-a791-b215fe968d56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-ced2f5fa-89da-4dde-a791-b215fe968d56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:10.273023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:10.273995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:10.274780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:10.274780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:10.274780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:10.274780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:10.274780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:10.274780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:10.274780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:10.274780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:10.274780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:10.274780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:10.274780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:10.274780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:10.279067 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 165/321] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:10.280437 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ea26f-012e-48ef-b6e8-9409e15f4c25 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:10.374182 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ea26f-012e-48ef-b6e8-9409e15f4c25 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:10.406150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe84e83-c6d4-48e6-a9d0-ff7b119ccdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:10.406820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe84e83-c6d4-48e6-a9d0-ff7b119ccdcf None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:10.407102 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe84e83-c6d4-48e6-a9d0-ff7b119ccdcf None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:10.407413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe84e83-c6d4-48e6-a9d0-ff7b119ccdcf None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:10.408157 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe84e83-c6d4-48e6-a9d0-ff7b119ccdcf None admin] RBAC: Adding query filter params (name=member) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:10.408464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe84e83-c6d4-48e6-a9d0-ff7b119ccdcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:10.410355 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe84e83-c6d4-48e6-a9d0-ff7b119ccdcf None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:10.418464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 157/322] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:10.423857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2f5fa-89da-4dde-a791-b215fe968d56 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:10.429103 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c343d-cac1-469c-9b9d-374e5f6fc9d1 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:10.429388 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c343d-cac1-469c-9b9d-374e5f6fc9d1 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:10.429660 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c343d-cac1-469c-9b9d-374e5f6fc9d1 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:10.431099 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 158/323] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:16:10.435769 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e1503-0c20-46eb-aac2-bb796e4d5f05 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:10.436052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e1503-0c20-46eb-aac2-bb796e4d5f05 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:10.436329 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e1503-0c20-46eb-aac2-bb796e4d5f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:10.451980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-5b8e1503-0c20-46eb-aac2-bb796e4d5f05 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:10.453136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-5b8e1503-0c20-46eb-aac2-bb796e4d5f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:10.516860 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 159/324] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:10.519897 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2f5fa-89da-4dde-a791-b215fe968d56 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:10.538643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e1503-0c20-46eb-aac2-bb796e4d5f05 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:10.553285 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfe34b-9505-4229-8eb0-df84c13ffb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:10.554163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfe34b-9505-4229-8eb0-df84c13ffb4b None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:10.554449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfe34b-9505-4229-8eb0-df84c13ffb4b None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:10.554723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfe34b-9505-4229-8eb0-df84c13ffb4b None admin] PATH_INFO: `/v3/users/c35b5d90d66c4fcb9059468d42cf0e05` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:10.555931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfe34b-9505-4229-8eb0-df84c13ffb4b None admin] RBAC: Authorizing `identity:get_user(user_id=c35b5d90d66c4fcb9059468d42cf0e05)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:10.557363 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfe34b-9505-4229-8eb0-df84c13ffb4b None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:10.563205 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 166/325] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:16:10 2022] GET /identity/v3/users/c35b5d90d66c4fcb9059468d42cf0e05 => generated 293 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:10.565610 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfe34b-9505-4229-8eb0-df84c13ffb4b None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:10.642438 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e1503-0c20-46eb-aac2-bb796e4d5f05 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:10.663222 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfe34b-9505-4229-8eb0-df84c13ffb4b None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:10.675865 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b765e-2a71-48f4-afa8-02fa9e8148f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:10.676669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b765e-2a71-48f4-afa8-02fa9e8148f2 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:10.676948 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b765e-2a71-48f4-afa8-02fa9e8148f2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:10.677223 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b765e-2a71-48f4-afa8-02fa9e8148f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:10.678379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b765e-2a71-48f4-afa8-02fa9e8148f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:10.679843 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b765e-2a71-48f4-afa8-02fa9e8148f2 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:10.681611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 160/326] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:10.687139 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b765e-2a71-48f4-afa8-02fa9e8148f2 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:10.695745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f175bd-7246-425e-a16d-33b92808ac09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:10.696339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f175bd-7246-425e-a16d-33b92808ac09 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:10.696618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f175bd-7246-425e-a16d-33b92808ac09 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:10.696889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f175bd-7246-425e-a16d-33b92808ac09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:10.697737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f175bd-7246-425e-a16d-33b92808ac09 None admin] RBAC: Adding query filter params (role.id=e1dddf0b88ee4578822ce51a2aba6760, user.id=c35b5d90d66c4fcb9059468d42cf0e05, scope.system=all) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:10.698035 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f175bd-7246-425e-a16d-33b92808ac09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:10.700158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f175bd-7246-425e-a16d-33b92808ac09 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:10.713062 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 167/327] 10.222.0.25 () {56 vars in 1308 bytes} [Mon May 9 18:16:10 2022] GET /identity/v3/role_assignments?user.id=c35b5d90d66c4fcb9059468d42cf0e05&scope.system=all&role.id=e1dddf0b88ee4578822ce51a2aba6760 => generated 510 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:10.760468 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ca598-eec6-4503-a046-83908abc9b8c None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:10.760774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ca598-eec6-4503-a046-83908abc9b8c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:10.761022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ca598-eec6-4503-a046-83908abc9b8c None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:10.762532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 168/328] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:16:10.767963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea3742-00a1-4e94-ae8d-17240cd003cb None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:10.768245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea3742-00a1-4e94-ae8d-17240cd003cb None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:10.768435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea3742-00a1-4e94-ae8d-17240cd003cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:10.788080 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-cbea3742-00a1-4e94-ae8d-17240cd003cb None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:10.789107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b765e-2a71-48f4-afa8-02fa9e8148f2 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:10.789304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-cbea3742-00a1-4e94-ae8d-17240cd003cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:10.824706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315342a3-9f64-42b4-938f-b082809835c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:10.825920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315342a3-9f64-42b4-938f-b082809835c0 None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:10.825920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315342a3-9f64-42b4-938f-b082809835c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:10.825920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315342a3-9f64-42b4-938f-b082809835c0 None admin] PATH_INFO: `/v3/groups` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:10.826593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315342a3-9f64-42b4-938f-b082809835c0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:10.828071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315342a3-9f64-42b4-938f-b082809835c0 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:10.845027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 161/329] 10.222.0.25 () {60 vars in 1134 bytes} [Mon May 9 18:16:10 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:16:10.861678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 169/330] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:10.886418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315342a3-9f64-42b4-938f-b082809835c0 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:10.996959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315342a3-9f64-42b4-938f-b082809835c0 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:11.030482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe39313-1266-4f21-aa95-029f22ce8b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:11.031224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe39313-1266-4f21-aa95-029f22ce8b61 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:11.031502 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe39313-1266-4f21-aa95-029f22ce8b61 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:11.031774 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe39313-1266-4f21-aa95-029f22ce8b61 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:11.042401 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe39313-1266-4f21-aa95-029f22ce8b61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-7fe39313-1266-4f21-aa95-029f22ce8b61 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:11.050069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:11.051246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:11.052246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:11.052246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:11.052246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:11.052246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:11.052246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:11.052246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:11.052246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:11.052246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:11.052246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:11.052246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:11.052246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:16:11.052246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:11.052246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 162/331] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:11.057027 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea3742-00a1-4e94-ae8d-17240cd003cb None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:11.156715 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea3742-00a1-4e94-ae8d-17240cd003cb None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:11.190160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81cf6b-f139-4312-9fb5-f366695d7bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:11.191351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81cf6b-f139-4312-9fb5-f366695d7bc2 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:11.191664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81cf6b-f139-4312-9fb5-f366695d7bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:11.191942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81cf6b-f139-4312-9fb5-f366695d7bc2 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:11.192679 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81cf6b-f139-4312-9fb5-f366695d7bc2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:11.192984 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81cf6b-f139-4312-9fb5-f366695d7bc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:11.194368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81cf6b-f139-4312-9fb5-f366695d7bc2 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:11.202699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 170/332] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:11.208873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe39313-1266-4f21-aa95-029f22ce8b61 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:11.309171 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe39313-1266-4f21-aa95-029f22ce8b61 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:11.343664 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74b72d-c402-4ab3-b038-6947312f177a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:11.344772 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74b72d-c402-4ab3-b038-6947312f177a None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:11.345189 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74b72d-c402-4ab3-b038-6947312f177a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:11.345621 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74b72d-c402-4ab3-b038-6947312f177a None admin] PATH_INFO: `/v3/users/a012a37135e443efb771b66b561863c4` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:11.346972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74b72d-c402-4ab3-b038-6947312f177a None admin] RBAC: Authorizing `identity:get_user(user_id=a012a37135e443efb771b66b561863c4)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:11.348569 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74b72d-c402-4ab3-b038-6947312f177a None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:11.350612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 163/333] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:16:11 2022] GET /identity/v3/users/a012a37135e443efb771b66b561863c4 => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:11.355995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81cf6b-f139-4312-9fb5-f366695d7bc2 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:11.463103 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81cf6b-f139-4312-9fb5-f366695d7bc2 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:11.497600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86c3a7-b5bd-49db-b47f-6518225a8e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:11.498463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86c3a7-b5bd-49db-b47f-6518225a8e72 None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:11.498745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86c3a7-b5bd-49db-b47f-6518225a8e72 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:11.499018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86c3a7-b5bd-49db-b47f-6518225a8e72 None admin] PATH_INFO: `/v3/system/users/a012a37135e443efb771b66b561863c4/roles/3c0125c8cabb462197565509a20404dc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:11.500518 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86c3a7-b5bd-49db-b47f-6518225a8e72 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a012a37135e443efb771b66b561863c4, role_id=3c0125c8cabb462197565509a20404dc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:11.501869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86c3a7-b5bd-49db-b47f-6518225a8e72 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:11.512056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 171/334] 10.222.0.25 () {58 vars in 1334 bytes} [Mon May 9 18:16:11 2022] PUT /identity/v3/system/users/a012a37135e443efb771b66b561863c4/roles/3c0125c8cabb462197565509a20404dc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:16:13.554493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9a311-ea6b-4b80-a46d-514e0ede5886 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:13.555544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9a311-ea6b-4b80-a46d-514e0ede5886 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:13.556137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9a311-ea6b-4b80-a46d-514e0ede5886 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:13.558020 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 164/335] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:13.567643 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63239fda-0830-4b6a-bfd5-6c16d2c96e2e None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:13.569267 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63239fda-0830-4b6a-bfd5-6c16d2c96e2e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:13.570184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63239fda-0830-4b6a-bfd5-6c16d2c96e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:13.593778 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-63239fda-0830-4b6a-bfd5-6c16d2c96e2e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:13.595306 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-63239fda-0830-4b6a-bfd5-6c16d2c96e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:13.677141 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 172/336] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:13.722802 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9a311-ea6b-4b80-a46d-514e0ede5886 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:13.739831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f2c19-b387-4090-bd78-294423ef4617 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:13.740428 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f2c19-b387-4090-bd78-294423ef4617 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:13.740897 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f2c19-b387-4090-bd78-294423ef4617 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:13.742429 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 173/337] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:13.747386 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fc6e4-63d7-4edf-a8cf-16a94697cbbf None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:13.747953 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fc6e4-63d7-4edf-a8cf-16a94697cbbf None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:13.748419 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fc6e4-63d7-4edf-a8cf-16a94697cbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:13.789315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-493fc6e4-63d7-4edf-a8cf-16a94697cbbf None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:13.796950 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-493fc6e4-63d7-4edf-a8cf-16a94697cbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:13.921372 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 174/338] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:13.948906 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9a311-ea6b-4b80-a46d-514e0ede5886 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:13.962759 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fc6e4-63d7-4edf-a8cf-16a94697cbbf None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:14.006860 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d7c54-fdc7-4305-baaf-2753affaa2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:14.007698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d7c54-fdc7-4305-baaf-2753affaa2e5 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:14.007976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d7c54-fdc7-4305-baaf-2753affaa2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:14.008245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d7c54-fdc7-4305-baaf-2753affaa2e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:14.032287 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d7c54-fdc7-4305-baaf-2753affaa2e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-da5d7c54-fdc7-4305-baaf-2753affaa2e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:14.040920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:14.041796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:14.042567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:14.042567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:14.042567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:14.042567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:14.042567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:14.042567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:14.042567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:14.042567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:14.042567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:14.042567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:14.042567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:14.042567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:14.047729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 165/339] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:16:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:14.050735 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d7c54-fdc7-4305-baaf-2753affaa2e5 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:14.099226 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fc6e4-63d7-4edf-a8cf-16a94697cbbf None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:14.154078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca53c5-0ab8-40d5-af9e-c85240288a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:14.154941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca53c5-0ab8-40d5-af9e-c85240288a13 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:14.155238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca53c5-0ab8-40d5-af9e-c85240288a13 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:14.155511 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca53c5-0ab8-40d5-af9e-c85240288a13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:14.187359 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d7c54-fdc7-4305-baaf-2753affaa2e5 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:14.188658 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca53c5-0ab8-40d5-af9e-c85240288a13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-9fca53c5-0ab8-40d5-af9e-c85240288a13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:14.196788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:14.197655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:14.198437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:14.198437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:14.198437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:14.198437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:14.198437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:14.198437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:14.198437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:14.198437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:14.198437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:14.198437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:14.198437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:14.198437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:14.205389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 175/340] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:14.211325 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca53c5-0ab8-40d5-af9e-c85240288a13 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:14.241572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fc734-2567-4c05-b586-b4ac079cf759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:14.242259 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fc734-2567-4c05-b586-b4ac079cf759 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:14.242543 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fc734-2567-4c05-b586-b4ac079cf759 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:14.242819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fc734-2567-4c05-b586-b4ac079cf759 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:14.243603 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fc734-2567-4c05-b586-b4ac079cf759 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:14.243915 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fc734-2567-4c05-b586-b4ac079cf759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:14.247202 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fc734-2567-4c05-b586-b4ac079cf759 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:14.262074 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 166/341] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:14.267012 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fc734-2567-4c05-b586-b4ac079cf759 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:14.384358 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fc734-2567-4c05-b586-b4ac079cf759 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:14.425808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe642d15-76b5-4da5-bc51-e5479d74606d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:14.426471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe642d15-76b5-4da5-bc51-e5479d74606d None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:14.426780 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe642d15-76b5-4da5-bc51-e5479d74606d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:14.427028 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe642d15-76b5-4da5-bc51-e5479d74606d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:14.440791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca53c5-0ab8-40d5-af9e-c85240288a13 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:14.501946 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8fcfb-8647-4c8c-ade5-0b718e10323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:14.502628 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8fcfb-8647-4c8c-ade5-0b718e10323f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:14.503237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8fcfb-8647-4c8c-ade5-0b718e10323f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:14.503510 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8fcfb-8647-4c8c-ade5-0b718e10323f None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:14.504334 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8fcfb-8647-4c8c-ade5-0b718e10323f None admin] RBAC: Adding query filter params (name=member) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:14.504559 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8fcfb-8647-4c8c-ade5-0b718e10323f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:14.505925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8fcfb-8647-4c8c-ade5-0b718e10323f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:14.517798 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe642d15-76b5-4da5-bc51-e5479d74606d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:14.519875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe642d15-76b5-4da5-bc51-e5479d74606d None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:14.521674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 176/342] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:14.523032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295eae88-3357-491b-a8c8-9c9383093a9d None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:14.523346 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295eae88-3357-491b-a8c8-9c9383093a9d None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:14.523622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295eae88-3357-491b-a8c8-9c9383093a9d None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:14.524963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 177/343] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:14.531402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6e8b-7d09-4521-954b-7e2e4914d039 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:14.531744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6e8b-7d09-4521-954b-7e2e4914d039 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:14.532027 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6e8b-7d09-4521-954b-7e2e4914d039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:14.556793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-1bfe6e8b-7d09-4521-954b-7e2e4914d039 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:14.558083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-1bfe6e8b-7d09-4521-954b-7e2e4914d039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:14.625358 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 167/344] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 18:16:14.626317 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe642d15-76b5-4da5-bc51-e5479d74606d None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:14.643453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 178/345] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:14.644855 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe6e8b-7d09-4521-954b-7e2e4914d039 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:14.727490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe642d15-76b5-4da5-bc51-e5479d74606d None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:14.747807 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe6e8b-7d09-4521-954b-7e2e4914d039 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:14.765883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e71561-2e4f-4f85-a63d-8076a79d34c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:14.766901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e71561-2e4f-4f85-a63d-8076a79d34c4 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:14.767444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e71561-2e4f-4f85-a63d-8076a79d34c4 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:14.767913 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e71561-2e4f-4f85-a63d-8076a79d34c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:14.782299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946648fb-40cc-4a3d-82a4-76daa04408e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:14.783150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946648fb-40cc-4a3d-82a4-76daa04408e4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:14.783456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946648fb-40cc-4a3d-82a4-76daa04408e4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:14.783729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946648fb-40cc-4a3d-82a4-76daa04408e4 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:14.788558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946648fb-40cc-4a3d-82a4-76daa04408e4 None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:14.788558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946648fb-40cc-4a3d-82a4-76daa04408e4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:14.788740 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 179/346] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:14 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:14.789430 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946648fb-40cc-4a3d-82a4-76daa04408e4 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:14.887876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946648fb-40cc-4a3d-82a4-76daa04408e4 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:14.916437 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e71561-2e4f-4f85-a63d-8076a79d34c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:14.918305 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e71561-2e4f-4f85-a63d-8076a79d34c4 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:14.925218 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78730910-a47f-459b-b9d9-9fa37dda26f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:14.926089 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78730910-a47f-459b-b9d9-9fa37dda26f2 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:14.926368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78730910-a47f-459b-b9d9-9fa37dda26f2 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:14.926639 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78730910-a47f-459b-b9d9-9fa37dda26f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:14.938232 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78730910-a47f-459b-b9d9-9fa37dda26f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-78730910-a47f-459b-b9d9-9fa37dda26f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:14.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:14.947798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:14.948604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:14.948604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:14.948604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:14.948604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:14.948604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:14.948604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:14.948604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:14.948604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:14.948604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:14.948604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:14.948604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:14.948604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:14.948604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 180/347] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:14.949274 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78730910-a47f-459b-b9d9-9fa37dda26f2 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:15.057462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78730910-a47f-459b-b9d9-9fa37dda26f2 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:15.092416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4cc5f-a68b-4277-ab07-29a68b790b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:15.093574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4cc5f-a68b-4277-ab07-29a68b790b74 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:15.094085 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4cc5f-a68b-4277-ab07-29a68b790b74 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:15.094454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4cc5f-a68b-4277-ab07-29a68b790b74 None admin] PATH_INFO: `/v3/groups/f92a13bf1cb3480aa636cced39bb1cee` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:15.101240 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4cc5f-a68b-4277-ab07-29a68b790b74 None admin] RBAC: Authorizing `identity:get_group(group_id=f92a13bf1cb3480aa636cced39bb1cee)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:15.102810 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4cc5f-a68b-4277-ab07-29a68b790b74 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:15.105063 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 181/348] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:14 2022] GET /identity/v3/groups/f92a13bf1cb3480aa636cced39bb1cee => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:15.105912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4cc5f-a68b-4277-ab07-29a68b790b74 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:15.131518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 168/349] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:15.132320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e71561-2e4f-4f85-a63d-8076a79d34c4 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:15.228413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4cc5f-a68b-4277-ab07-29a68b790b74 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:15.243446 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e71561-2e4f-4f85-a63d-8076a79d34c4 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:15.280068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af38a8-0f0c-4cc3-88d1-fec0550df6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:15.281064 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af38a8-0f0c-4cc3-88d1-fec0550df6a1 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:15.281351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af38a8-0f0c-4cc3-88d1-fec0550df6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:15.281627 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af38a8-0f0c-4cc3-88d1-fec0550df6a1 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:15.282658 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af38a8-0f0c-4cc3-88d1-fec0550df6a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:15.282970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af38a8-0f0c-4cc3-88d1-fec0550df6a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:15.285393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af38a8-0f0c-4cc3-88d1-fec0550df6a1 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:15.289649 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440ba36-a605-4d92-8d34-dc758a5f326c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:15.290316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440ba36-a605-4d92-8d34-dc758a5f326c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:15.290595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440ba36-a605-4d92-8d34-dc758a5f326c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:15.290869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440ba36-a605-4d92-8d34-dc758a5f326c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:15.291725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440ba36-a605-4d92-8d34-dc758a5f326c None admin] RBAC: Adding query filter params (group.id=f92a13bf1cb3480aa636cced39bb1cee, role.id=9d45ca30c3584deabd69f2cd196ae5cc, scope.project.id=36617c1516b14f15a3b4a35692ff654f) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:15.292098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440ba36-a605-4d92-8d34-dc758a5f326c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:15.293564 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440ba36-a605-4d92-8d34-dc758a5f326c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:15.295016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 182/350] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:15.296377 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c1385-d7fb-4ae8-ab02-4afb74b7ead4 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:15.296662 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c1385-d7fb-4ae8-ab02-4afb74b7ead4 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:15.296935 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c1385-d7fb-4ae8-ab02-4afb74b7ead4 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:15.298373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 183/351] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:15.299023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c1385-d7fb-4ae8-ab02-4afb74b7ead4 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:15.306051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 169/352] 10.222.0.25 () {56 vars in 1376 bytes} [Mon May 9 18:16:15 2022] GET /identity/v3/role_assignments?group.id=f92a13bf1cb3480aa636cced39bb1cee&scope.project.id=36617c1516b14f15a3b4a35692ff654f&role.id=9d45ca30c3584deabd69f2cd196ae5cc => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:15.307197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440ba36-a605-4d92-8d34-dc758a5f326c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:15.426897 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440ba36-a605-4d92-8d34-dc758a5f326c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:15.441943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c1385-d7fb-4ae8-ab02-4afb74b7ead4 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:15.475282 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b63a2-318e-42d3-a571-db88df3c9dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:15.475966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b63a2-318e-42d3-a571-db88df3c9dee None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:15.476418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b63a2-318e-42d3-a571-db88df3c9dee None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:15.476710 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b63a2-318e-42d3-a571-db88df3c9dee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:15.482457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6cb95-9e00-4086-9154-ed4d933a7da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:15.483733 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6cb95-9e00-4086-9154-ed4d933a7da3 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:15.484237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6cb95-9e00-4086-9154-ed4d933a7da3 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:15.484729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6cb95-9e00-4086-9154-ed4d933a7da3 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:15.486181 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6cb95-9e00-4086-9154-ed4d933a7da3 None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:15.487947 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6cb95-9e00-4086-9154-ed4d933a7da3 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:15.489955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 184/353] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:15 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:15.491430 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0304d-b35f-42a6-8acb-85218eaa50d9 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:15.491945 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0304d-b35f-42a6-8acb-85218eaa50d9 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:15.492457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0304d-b35f-42a6-8acb-85218eaa50d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:15.511160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-d3e0304d-b35f-42a6-8acb-85218eaa50d9 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:15.513237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-d3e0304d-b35f-42a6-8acb-85218eaa50d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:15.558885 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b63a2-318e-42d3-a571-db88df3c9dee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:15.560750 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b63a2-318e-42d3-a571-db88df3c9dee None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:15.587881 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 185/354] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:15.589693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710767a-fa61-466a-9248-1348742028fe None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:15.590175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710767a-fa61-466a-9248-1348742028fe None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:15.590688 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710767a-fa61-466a-9248-1348742028fe None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:15.592195 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 186/355] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:15.593103 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710767a-fa61-466a-9248-1348742028fe None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:15.670233 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 170/356] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:15.673167 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b40ad-9316-48e6-b150-832d4fc7ba98 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:15.673167 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b40ad-9316-48e6-b150-832d4fc7ba98 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:15.673167 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b40ad-9316-48e6-b150-832d4fc7ba98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:15.692311 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-c43b40ad-9316-48e6-b150-832d4fc7ba98 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:15.693481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c43b40ad-9316-48e6-b150-832d4fc7ba98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:15.739210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710767a-fa61-466a-9248-1348742028fe None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:15.772757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 171/357] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:15.773619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b40ad-9316-48e6-b150-832d4fc7ba98 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:15.793719 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de625f-4534-4cde-ba29-a986ec11d550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:15.794575 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de625f-4534-4cde-ba29-a986ec11d550 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:15.794850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de625f-4534-4cde-ba29-a986ec11d550 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:15.795120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de625f-4534-4cde-ba29-a986ec11d550 None admin] PATH_INFO: `/v3/groups/f92a13bf1cb3480aa636cced39bb1cee` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:15.796359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de625f-4534-4cde-ba29-a986ec11d550 None admin] RBAC: Authorizing `identity:get_group(group_id=f92a13bf1cb3480aa636cced39bb1cee)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:15.797800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de625f-4534-4cde-ba29-a986ec11d550 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:15.801186 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 187/358] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:15 2022] GET /identity/v3/groups/f92a13bf1cb3480aa636cced39bb1cee => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:15.804256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397fd8d-b76f-4595-8a85-8b230240df5a None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:15.804543 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397fd8d-b76f-4595-8a85-8b230240df5a None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:15.805797 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397fd8d-b76f-4595-8a85-8b230240df5a None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:15.807097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 188/359] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:15.808955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397fd8d-b76f-4595-8a85-8b230240df5a None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:15.914989 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b40ad-9316-48e6-b150-832d4fc7ba98 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:15.932963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397fd8d-b76f-4595-8a85-8b230240df5a None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:15.961339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0360093-02c7-4ebb-9c18-0f838b927d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:15.962078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0360093-02c7-4ebb-9c18-0f838b927d08 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:15.962358 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0360093-02c7-4ebb-9c18-0f838b927d08 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:15.962635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0360093-02c7-4ebb-9c18-0f838b927d08 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:15.976929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb9742-300d-4492-aa06-52c496acc1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:15.977793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb9742-300d-4492-aa06-52c496acc1a5 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:15.978088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0360093-02c7-4ebb-9c18-0f838b927d08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:15.978197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb9742-300d-4492-aa06-52c496acc1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:15.978393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb9742-300d-4492-aa06-52c496acc1a5 None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:15.980186 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb9742-300d-4492-aa06-52c496acc1a5 None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:15.981008 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb9742-300d-4492-aa06-52c496acc1a5 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:15.982838 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 189/360] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:15 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:15.984330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb9742-300d-4492-aa06-52c496acc1a5 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-c0360093-02c7-4ebb-9c18-0f838b927d08 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:15.986301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:15.987241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:15.988145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:15.988145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:15.988145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:15.988145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:15.988145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:15.988145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:15.988145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:15.988145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:15.988145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:15.988145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:15.988145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:15.988145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:15.989263 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 172/361] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:16:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:15.989926 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0360093-02c7-4ebb-9c18-0f838b927d08 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:16.099113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb9742-300d-4492-aa06-52c496acc1a5 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:16.102926 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0360093-02c7-4ebb-9c18-0f838b927d08 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:16.142689 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511d09f-b716-42e7-a81e-ce0e0dd938e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:16.142904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fed9bf-67a4-48cb-95e0-a13c25db7685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:16.142904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fed9bf-67a4-48cb-95e0-a13c25db7685 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:16.143413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fed9bf-67a4-48cb-95e0-a13c25db7685 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:16.143536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fed9bf-67a4-48cb-95e0-a13c25db7685 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:16.145150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511d09f-b716-42e7-a81e-ce0e0dd938e2 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:16.145643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511d09f-b716-42e7-a81e-ce0e0dd938e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:16.146125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511d09f-b716-42e7-a81e-ce0e0dd938e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:16.147166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511d09f-b716-42e7-a81e-ce0e0dd938e2 None admin] RBAC: Adding query filter params (group.id=f92a13bf1cb3480aa636cced39bb1cee, role.id=e1dddf0b88ee4578822ce51a2aba6760, scope.project.id=36617c1516b14f15a3b4a35692ff654f) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:16.147721 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511d09f-b716-42e7-a81e-ce0e0dd938e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:16.149365 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511d09f-b716-42e7-a81e-ce0e0dd938e2 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:16.154681 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fed9bf-67a4-48cb-95e0-a13c25db7685 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:16.155041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 173/362] 10.222.0.25 () {56 vars in 1376 bytes} [Mon May 9 18:16:15 2022] GET /identity/v3/role_assignments?group.id=f92a13bf1cb3480aa636cced39bb1cee&scope.project.id=36617c1516b14f15a3b4a35692ff654f&role.id=e1dddf0b88ee4578822ce51a2aba6760 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:16.156368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa82d7-aeb2-41f9-951c-94c11ada662f None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:16.156657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa82d7-aeb2-41f9-951c-94c11ada662f None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:16.156937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa82d7-aeb2-41f9-951c-94c11ada662f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-c8fed9bf-67a4-48cb-95e0-a13c25db7685 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:16.163579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:16.164574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:16.165391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:16.165391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:16.165391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:16.165391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:16.165391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:16.165391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:16.165391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:16.165391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:16.165391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:16.165391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:16.165391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 18:16:16.165391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:16.165391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 190/363] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:16.166086 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fed9bf-67a4-48cb-95e0-a13c25db7685 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:16.177650 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-a1fa82d7-aeb2-41f9-951c-94c11ada662f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:16.179286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a1fa82d7-aeb2-41f9-951c-94c11ada662f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:16.283680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 174/364] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:16.285199 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa82d7-aeb2-41f9-951c-94c11ada662f None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:16.327238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fed9bf-67a4-48cb-95e0-a13c25db7685 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:16.370093 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c7ab0-edee-467c-b651-73c1a38a10ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:16.370978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c7ab0-edee-467c-b651-73c1a38a10ba None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:16.371303 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c7ab0-edee-467c-b651-73c1a38a10ba None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:16.371582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c7ab0-edee-467c-b651-73c1a38a10ba None admin] PATH_INFO: `/v3/groups/f92a13bf1cb3480aa636cced39bb1cee` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:16.372987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c7ab0-edee-467c-b651-73c1a38a10ba None admin] RBAC: Authorizing `identity:get_group(group_id=f92a13bf1cb3480aa636cced39bb1cee)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:16.375322 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c7ab0-edee-467c-b651-73c1a38a10ba None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:16.377258 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 191/365] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:16 2022] GET /identity/v3/groups/f92a13bf1cb3480aa636cced39bb1cee => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:16.379782 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c7ab0-edee-467c-b651-73c1a38a10ba None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:16.416196 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa82d7-aeb2-41f9-951c-94c11ada662f None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:16.465502 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f492-e6ee-494e-b8eb-ff26994bf500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:16.466187 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f492-e6ee-494e-b8eb-ff26994bf500 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:16.466470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f492-e6ee-494e-b8eb-ff26994bf500 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:16.466745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f492-e6ee-494e-b8eb-ff26994bf500 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:16.467884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4f492-e6ee-494e-b8eb-ff26994bf500 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:16.468191 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4f492-e6ee-494e-b8eb-ff26994bf500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:16.470168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4f492-e6ee-494e-b8eb-ff26994bf500 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:16.480821 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 175/366] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:16.481640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f492-e6ee-494e-b8eb-ff26994bf500 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:16.493254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c7ab0-edee-467c-b651-73c1a38a10ba None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:16.538527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be493a90-3adc-4cf3-9a12-c9072ebf06a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:16.539130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be493a90-3adc-4cf3-9a12-c9072ebf06a2 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:16.539439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be493a90-3adc-4cf3-9a12-c9072ebf06a2 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:16.540124 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be493a90-3adc-4cf3-9a12-c9072ebf06a2 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:16.541155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be493a90-3adc-4cf3-9a12-c9072ebf06a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:16.541466 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be493a90-3adc-4cf3-9a12-c9072ebf06a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:16.543843 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be493a90-3adc-4cf3-9a12-c9072ebf06a2 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:16.554557 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 192/367] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:16.555532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be493a90-3adc-4cf3-9a12-c9072ebf06a2 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:16.610157 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f492-e6ee-494e-b8eb-ff26994bf500 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:16.659375 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdec51-6682-4905-822e-5a52a4a21835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:16.661153 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdec51-6682-4905-822e-5a52a4a21835 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:16.661446 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdec51-6682-4905-822e-5a52a4a21835 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:16.661729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdec51-6682-4905-822e-5a52a4a21835 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:16.676970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cdec51-6682-4905-822e-5a52a4a21835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-98cdec51-6682-4905-822e-5a52a4a21835 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:16.686699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:16.687599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:16.688386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:16.688386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:16.688386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:16.688386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:16.688386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:16.688386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:16.688386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:16.688386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:16.688386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:16.688386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:16.688386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:16:16.688386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:16.688977 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 176/368] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:16:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:16.689566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdec51-6682-4905-822e-5a52a4a21835 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:16.707106 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be493a90-3adc-4cf3-9a12-c9072ebf06a2 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:16.749092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649302df-6dfa-4aca-b042-2366baca8311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:16.749839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649302df-6dfa-4aca-b042-2366baca8311 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:16.750080 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649302df-6dfa-4aca-b042-2366baca8311 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:16.750363 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649302df-6dfa-4aca-b042-2366baca8311 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:16.751206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649302df-6dfa-4aca-b042-2366baca8311 None admin] RBAC: Adding query filter params (group.id=f92a13bf1cb3480aa636cced39bb1cee, role.id=e1dddf0b88ee4578822ce51a2aba6760, scope.project.id=82a98033ef0f45f887ea92601788b835) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:16.751577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649302df-6dfa-4aca-b042-2366baca8311 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:16.753914 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649302df-6dfa-4aca-b042-2366baca8311 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:16.762489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 193/369] 10.222.0.25 () {56 vars in 1376 bytes} [Mon May 9 18:16:16 2022] GET /identity/v3/role_assignments?group.id=f92a13bf1cb3480aa636cced39bb1cee&scope.project.id=82a98033ef0f45f887ea92601788b835&role.id=e1dddf0b88ee4578822ce51a2aba6760 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:16.763512 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649302df-6dfa-4aca-b042-2366baca8311 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:16.800401 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdec51-6682-4905-822e-5a52a4a21835 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:16.835741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de1a8a-c81b-4204-afc2-83cba670995c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:16.836406 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1a8a-c81b-4204-afc2-83cba670995c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:16.836690 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1a8a-c81b-4204-afc2-83cba670995c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:16.836965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1a8a-c81b-4204-afc2-83cba670995c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:16.876876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649302df-6dfa-4aca-b042-2366baca8311 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:16.910015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de1a8a-c81b-4204-afc2-83cba670995c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:16.911517 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de1a8a-c81b-4204-afc2-83cba670995c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:16.918803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414534bd-c2a3-4aba-a2fb-93ed4543aa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:16.919718 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414534bd-c2a3-4aba-a2fb-93ed4543aa94 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:16.920001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414534bd-c2a3-4aba-a2fb-93ed4543aa94 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:16.920276 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414534bd-c2a3-4aba-a2fb-93ed4543aa94 None admin] PATH_INFO: `/v3/users/a012a37135e443efb771b66b561863c4` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:16.921480 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414534bd-c2a3-4aba-a2fb-93ed4543aa94 None admin] RBAC: Authorizing `identity:get_user(user_id=a012a37135e443efb771b66b561863c4)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:16.923036 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414534bd-c2a3-4aba-a2fb-93ed4543aa94 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:16.930022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 194/370] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:16:16 2022] GET /identity/v3/users/a012a37135e443efb771b66b561863c4 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:16.930848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414534bd-c2a3-4aba-a2fb-93ed4543aa94 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:17.018008 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 177/371] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:17.018892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de1a8a-c81b-4204-afc2-83cba670995c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:17.087695 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414534bd-c2a3-4aba-a2fb-93ed4543aa94 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:17.121376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de1a8a-c81b-4204-afc2-83cba670995c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:17.131116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d7ff1-2c4d-4a84-ad55-ed94dedbe65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:17.132006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d7ff1-2c4d-4a84-ad55-ed94dedbe65f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:17.132297 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d7ff1-2c4d-4a84-ad55-ed94dedbe65f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:17.132577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d7ff1-2c4d-4a84-ad55-ed94dedbe65f None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:17.133336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d7ff1-2c4d-4a84-ad55-ed94dedbe65f None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:17.133643 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d7ff1-2c4d-4a84-ad55-ed94dedbe65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:17.135019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d7ff1-2c4d-4a84-ad55-ed94dedbe65f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:17.148926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 195/372] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:17.149883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d7ff1-2c4d-4a84-ad55-ed94dedbe65f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:17.166077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabd035-f948-4254-bc9d-4f558d8932bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:17.166774 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabd035-f948-4254-bc9d-4f558d8932bc None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:17.167059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabd035-f948-4254-bc9d-4f558d8932bc None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:17.167375 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabd035-f948-4254-bc9d-4f558d8932bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:17.168216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabd035-f948-4254-bc9d-4f558d8932bc None admin] RBAC: Adding query filter params (role.id=3c0125c8cabb462197565509a20404dc, user.id=a012a37135e443efb771b66b561863c4, scope.system=all) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:17.168529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabd035-f948-4254-bc9d-4f558d8932bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:17.169976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabd035-f948-4254-bc9d-4f558d8932bc None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:17.183825 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 178/373] 10.222.0.25 () {56 vars in 1308 bytes} [Mon May 9 18:16:17 2022] GET /identity/v3/role_assignments?user.id=a012a37135e443efb771b66b561863c4&scope.system=all&role.id=3c0125c8cabb462197565509a20404dc => generated 510 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:17.184703 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabd035-f948-4254-bc9d-4f558d8932bc None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:17.305372 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d7ff1-2c4d-4a84-ad55-ed94dedbe65f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:17.365144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fc808-9824-4f8f-81fb-99875b7f8266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:17.365948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fc808-9824-4f8f-81fb-99875b7f8266 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:17.366225 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fc808-9824-4f8f-81fb-99875b7f8266 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:17.366496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fc808-9824-4f8f-81fb-99875b7f8266 None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:17.368446 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2fc808-9824-4f8f-81fb-99875b7f8266 None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:17.369874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2fc808-9824-4f8f-81fb-99875b7f8266 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:17.373332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 196/374] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:17 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:17.378651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fc808-9824-4f8f-81fb-99875b7f8266 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:17.403321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabd035-f948-4254-bc9d-4f558d8932bc None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:17.484747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4216d3e-f69f-4f40-b55f-fd79e5dc9794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:17.485427 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4216d3e-f69f-4f40-b55f-fd79e5dc9794 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:17.485716 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4216d3e-f69f-4f40-b55f-fd79e5dc9794 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:17.485981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4216d3e-f69f-4f40-b55f-fd79e5dc9794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:17.572149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fc808-9824-4f8f-81fb-99875b7f8266 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:17.603137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4216d3e-f69f-4f40-b55f-fd79e5dc9794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:17.604827 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4216d3e-f69f-4f40-b55f-fd79e5dc9794 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:17.657089 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a125c1-9b59-4c8c-9d4a-1c2569f6ebe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:17.657902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a125c1-9b59-4c8c-9d4a-1c2569f6ebe0 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:17.658187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a125c1-9b59-4c8c-9d4a-1c2569f6ebe0 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:17.658467 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a125c1-9b59-4c8c-9d4a-1c2569f6ebe0 None admin] PATH_INFO: `/v3/groups/f92a13bf1cb3480aa636cced39bb1cee` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:17.664242 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a125c1-9b59-4c8c-9d4a-1c2569f6ebe0 None admin] RBAC: Authorizing `identity:get_group(group_id=f92a13bf1cb3480aa636cced39bb1cee)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:17.665712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a125c1-9b59-4c8c-9d4a-1c2569f6ebe0 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:17.667440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 197/375] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:17 2022] GET /identity/v3/groups/f92a13bf1cb3480aa636cced39bb1cee => generated 226 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:17.687456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a125c1-9b59-4c8c-9d4a-1c2569f6ebe0 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:17.721803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 179/376] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:17.729101 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4216d3e-f69f-4f40-b55f-fd79e5dc9794 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:17.885380 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a125c1-9b59-4c8c-9d4a-1c2569f6ebe0 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:17.888352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4216d3e-f69f-4f40-b55f-fd79e5dc9794 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:17.932290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6f54d-f3a3-49c6-8dba-819103729adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:17.932290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6f54d-f3a3-49c6-8dba-819103729adf None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:17.932290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6f54d-f3a3-49c6-8dba-819103729adf None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:17.932598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6f54d-f3a3-49c6-8dba-819103729adf None admin] PATH_INFO: `/v3/projects/969d4448da94407dbcabadb61787000d` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:17.939964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a6f54d-f3a3-49c6-8dba-819103729adf None admin] RBAC: Authorizing `identity:get_project(project_id=969d4448da94407dbcabadb61787000d)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:17.942534 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908aa10-e8e6-4943-893c-2bb6d7239874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:17.942681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a6f54d-f3a3-49c6-8dba-819103729adf None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:17.943575 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908aa10-e8e6-4943-893c-2bb6d7239874 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:17.944068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908aa10-e8e6-4943-893c-2bb6d7239874 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:17.944532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908aa10-e8e6-4943-893c-2bb6d7239874 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:17.945561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908aa10-e8e6-4943-893c-2bb6d7239874 None admin] RBAC: Adding query filter params (group.id=f92a13bf1cb3480aa636cced39bb1cee, role.id=9d45ca30c3584deabd69f2cd196ae5cc, scope.project.id=82a98033ef0f45f887ea92601788b835) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:17.945666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 180/377] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:17 2022] GET /identity/v3/projects/969d4448da94407dbcabadb61787000d => generated 344 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:17.946148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908aa10-e8e6-4943-893c-2bb6d7239874 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:17.947784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908aa10-e8e6-4943-893c-2bb6d7239874 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:17.963289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 198/378] 10.222.0.25 () {56 vars in 1376 bytes} [Mon May 9 18:16:17 2022] GET /identity/v3/role_assignments?group.id=f92a13bf1cb3480aa636cced39bb1cee&scope.project.id=82a98033ef0f45f887ea92601788b835&role.id=9d45ca30c3584deabd69f2cd196ae5cc => generated 260 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:17.971978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6f54d-f3a3-49c6-8dba-819103729adf None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:18.022135 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241afa5-374e-4f85-8861-3c1874c64bec None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:18.022457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241afa5-374e-4f85-8861-3c1874c64bec None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:18.022690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241afa5-374e-4f85-8861-3c1874c64bec None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:18.024341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 199/379] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:16:18.038546 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9639cd-c9e4-4cdb-8e38-1a4316fa197c None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:18.038869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9639cd-c9e4-4cdb-8e38-1a4316fa197c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:18.039102 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9639cd-c9e4-4cdb-8e38-1a4316fa197c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:18.070136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-6b9639cd-c9e4-4cdb-8e38-1a4316fa197c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:18.071438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-6b9639cd-c9e4-4cdb-8e38-1a4316fa197c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:18.108763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6f54d-f3a3-49c6-8dba-819103729adf None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:18.150356 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7fe71-59e6-4901-9e59-ab2ed940e798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:18.151258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7fe71-59e6-4901-9e59-ab2ed940e798 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:18.151708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7fe71-59e6-4901-9e59-ab2ed940e798 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:18.152150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7fe71-59e6-4901-9e59-ab2ed940e798 None admin] PATH_INFO: `/v3/groups/aa85b36cfe21438ca82f196c6d2e6f78` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:18.163435 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7fe71-59e6-4901-9e59-ab2ed940e798 None admin] RBAC: Authorizing `identity:get_group(group_id=aa85b36cfe21438ca82f196c6d2e6f78)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:18.170083 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7fe71-59e6-4901-9e59-ab2ed940e798 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:18.172164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 181/380] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:17 2022] GET /identity/v3/groups/aa85b36cfe21438ca82f196c6d2e6f78 => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:18.179591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7fe71-59e6-4901-9e59-ab2ed940e798 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:18.212058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 200/381] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:18.233848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9639cd-c9e4-4cdb-8e38-1a4316fa197c None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:18.320809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7fe71-59e6-4901-9e59-ab2ed940e798 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:18.333619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9639cd-c9e4-4cdb-8e38-1a4316fa197c None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:18.363769 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11d668-d9aa-4166-aacd-a762169fcf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:18.364711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11d668-d9aa-4166-aacd-a762169fcf3e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:18.365151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11d668-d9aa-4166-aacd-a762169fcf3e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:18.365570 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11d668-d9aa-4166-aacd-a762169fcf3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:18.366555 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11d668-d9aa-4166-aacd-a762169fcf3e None admin] RBAC: Adding query filter params (group.id=aa85b36cfe21438ca82f196c6d2e6f78, role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=969d4448da94407dbcabadb61787000d) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:18.367018 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11d668-d9aa-4166-aacd-a762169fcf3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:18.368615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11d668-d9aa-4166-aacd-a762169fcf3e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:18.368729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d8b31-3539-4042-aca3-cfa12f760a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:18.369492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d8b31-3539-4042-aca3-cfa12f760a3f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:18.369947 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d8b31-3539-4042-aca3-cfa12f760a3f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:18.370358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d8b31-3539-4042-aca3-cfa12f760a3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:18.375555 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 182/382] 10.222.0.25 () {56 vars in 1376 bytes} [Mon May 9 18:16:18 2022] GET /identity/v3/role_assignments?group.id=aa85b36cfe21438ca82f196c6d2e6f78&scope.project.id=969d4448da94407dbcabadb61787000d&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:18.383195 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d8b31-3539-4042-aca3-cfa12f760a3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-f44d8b31-3539-4042-aca3-cfa12f760a3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:18.391942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:18.393105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:18.393938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:18.393938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:18.393938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:18.393938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:18.393938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:18.393938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:18.393938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:18.393938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:18.393938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:18.393938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:18.393938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:18.393938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:18.396037 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 201/383] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:16:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:18.402013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11d668-d9aa-4166-aacd-a762169fcf3e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:18.521531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11d668-d9aa-4166-aacd-a762169fcf3e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:18.566438 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a001e-b611-46e4-9724-d6a46cd9a9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:18.567415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a001e-b611-46e4-9724-d6a46cd9a9b7 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:18.567866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a001e-b611-46e4-9724-d6a46cd9a9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:18.568311 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a001e-b611-46e4-9724-d6a46cd9a9b7 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:18.569237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a001e-b611-46e4-9724-d6a46cd9a9b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:18.569691 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a001e-b611-46e4-9724-d6a46cd9a9b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:18.571245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a001e-b611-46e4-9724-d6a46cd9a9b7 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:18.581340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 183/384] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:18.586743 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d8b31-3539-4042-aca3-cfa12f760a3f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:18.717888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d8b31-3539-4042-aca3-cfa12f760a3f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:18.751962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7721e6-5043-40cf-a8ab-61745a9f5817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:18.753109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7721e6-5043-40cf-a8ab-61745a9f5817 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:18.753386 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7721e6-5043-40cf-a8ab-61745a9f5817 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:18.753658 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7721e6-5043-40cf-a8ab-61745a9f5817 None admin] PATH_INFO: `/v3/groups/f92a13bf1cb3480aa636cced39bb1cee` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:18.754814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7721e6-5043-40cf-a8ab-61745a9f5817 None admin] RBAC: Authorizing `identity:get_group(group_id=f92a13bf1cb3480aa636cced39bb1cee)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:18.756279 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7721e6-5043-40cf-a8ab-61745a9f5817 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:18.758164 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 202/385] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:18 2022] GET /identity/v3/groups/f92a13bf1cb3480aa636cced39bb1cee => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:18.763618 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a001e-b611-46e4-9724-d6a46cd9a9b7 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:18.871539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a001e-b611-46e4-9724-d6a46cd9a9b7 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:18.908935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4196f-dead-4bdf-bee2-66c847cbbee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:18.910348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4196f-dead-4bdf-bee2-66c847cbbee0 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:18.910622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4196f-dead-4bdf-bee2-66c847cbbee0 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:18.910889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4196f-dead-4bdf-bee2-66c847cbbee0 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:18.912079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4196f-dead-4bdf-bee2-66c847cbbee0 None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:18.913518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4196f-dead-4bdf-bee2-66c847cbbee0 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:18.915213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 184/386] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:18 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:18.920938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7721e6-5043-40cf-a8ab-61745a9f5817 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:19.039546 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7721e6-5043-40cf-a8ab-61745a9f5817 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:19.082135 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf4794-0a51-4670-90e4-47c43bd5cb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:19.083022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf4794-0a51-4670-90e4-47c43bd5cb85 None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:19.083350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf4794-0a51-4670-90e4-47c43bd5cb85 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:19.083620 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf4794-0a51-4670-90e4-47c43bd5cb85 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f/groups/f92a13bf1cb3480aa636cced39bb1cee/roles/9d45ca30c3584deabd69f2cd196ae5cc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:19.085853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf4794-0a51-4670-90e4-47c43bd5cb85 None admin] RBAC: Authorizing `identity:create_grant(project_id=36617c1516b14f15a3b4a35692ff654f, group_id=f92a13bf1cb3480aa636cced39bb1cee, role_id=9d45ca30c3584deabd69f2cd196ae5cc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:19.087298 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf4794-0a51-4670-90e4-47c43bd5cb85 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:19.107358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 203/387] 10.222.0.25 () {58 vars in 1442 bytes} [Mon May 9 18:16:18 2022] PUT /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f/groups/f92a13bf1cb3480aa636cced39bb1cee/roles/9d45ca30c3584deabd69f2cd196ae5cc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:16:19.344059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3521317-f943-425e-a142-a9fcb492d187 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:19.344449 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3521317-f943-425e-a142-a9fcb492d187 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:19.344640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3521317-f943-425e-a142-a9fcb492d187 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:19.346226 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 185/388] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:19.351960 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ff5f3-4649-43d4-9fbc-662ce31621da None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:19.352247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ff5f3-4649-43d4-9fbc-662ce31621da None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:19.352524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ff5f3-4649-43d4-9fbc-662ce31621da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:19.371712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-039ff5f3-4649-43d4-9fbc-662ce31621da None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:19.373629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-039ff5f3-4649-43d4-9fbc-662ce31621da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:19.454684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 204/389] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:19.492134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3521317-f943-425e-a142-a9fcb492d187 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:19.604590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3521317-f943-425e-a142-a9fcb492d187 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:19.643233 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08654b-42ff-47de-8158-925c6867dad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:19.644606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08654b-42ff-47de-8158-925c6867dad0 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:19.644933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08654b-42ff-47de-8158-925c6867dad0 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:19.645250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08654b-42ff-47de-8158-925c6867dad0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:19.655683 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08654b-42ff-47de-8158-925c6867dad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-0a08654b-42ff-47de-8158-925c6867dad0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:19.663509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:19.664361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:19.665927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:19.665927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:19.665927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:19.665927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:19.665927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:19.665927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:19.665927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:19.665927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:19.665927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:19.665927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:19.665927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:19.665927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:19.665927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 186/390] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:19.670767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ff5f3-4649-43d4-9fbc-662ce31621da None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:19.813461 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ff5f3-4649-43d4-9fbc-662ce31621da None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:19.855710 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13e442-692b-408c-bd65-73fd3b8359ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:19.856787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13e442-692b-408c-bd65-73fd3b8359ad None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:19.857280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13e442-692b-408c-bd65-73fd3b8359ad None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:19.857766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13e442-692b-408c-bd65-73fd3b8359ad None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:19.859169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13e442-692b-408c-bd65-73fd3b8359ad None admin] RBAC: Adding query filter params (name=member) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:19.859779 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13e442-692b-408c-bd65-73fd3b8359ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:19.862893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13e442-692b-408c-bd65-73fd3b8359ad None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:19.878971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 205/391] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:19.884610 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08654b-42ff-47de-8158-925c6867dad0 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:19.923834 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efa344-0174-4fba-bd1b-3fa5f64bf02d None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:19.924481 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efa344-0174-4fba-bd1b-3fa5f64bf02d None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:19.924481 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efa344-0174-4fba-bd1b-3fa5f64bf02d None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:19.926889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 206/392] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:19.932978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a398398-e5db-4c8d-955e-d585929bda73 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:19.933485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a398398-e5db-4c8d-955e-d585929bda73 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:19.933964 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a398398-e5db-4c8d-955e-d585929bda73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:19.962276 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4a398398-e5db-4c8d-955e-d585929bda73 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:19.962276 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4a398398-e5db-4c8d-955e-d585929bda73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:20.016734 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08654b-42ff-47de-8158-925c6867dad0 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:20.051819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efafbf8-42ed-4bd8-9b75-5ae19ef2a94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:20.052537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 207/393] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:20.052736 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efafbf8-42ed-4bd8-9b75-5ae19ef2a94e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:20.053021 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efafbf8-42ed-4bd8-9b75-5ae19ef2a94e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:20.053396 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efafbf8-42ed-4bd8-9b75-5ae19ef2a94e None admin] PATH_INFO: `/v3/groups/f92a13bf1cb3480aa636cced39bb1cee` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:20.054616 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efafbf8-42ed-4bd8-9b75-5ae19ef2a94e None admin] RBAC: Authorizing `identity:get_group(group_id=f92a13bf1cb3480aa636cced39bb1cee)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:20.056097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efafbf8-42ed-4bd8-9b75-5ae19ef2a94e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:20.057983 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 187/394] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:19 2022] GET /identity/v3/groups/f92a13bf1cb3480aa636cced39bb1cee => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:20.063024 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a398398-e5db-4c8d-955e-d585929bda73 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:20.076327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efafbf8-42ed-4bd8-9b75-5ae19ef2a94e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:20.165810 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a398398-e5db-4c8d-955e-d585929bda73 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:20.196369 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efafbf8-42ed-4bd8-9b75-5ae19ef2a94e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:20.201222 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dcbfd-3c10-4744-b8d6-dbe8f9f99d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:20.202083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dcbfd-3c10-4744-b8d6-dbe8f9f99d57 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:20.202355 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dcbfd-3c10-4744-b8d6-dbe8f9f99d57 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:20.202621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dcbfd-3c10-4744-b8d6-dbe8f9f99d57 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:20.203817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016dcbfd-3c10-4744-b8d6-dbe8f9f99d57 None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:20.205241 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016dcbfd-3c10-4744-b8d6-dbe8f9f99d57 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:20.207247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 208/395] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:20 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:20.212792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dcbfd-3c10-4744-b8d6-dbe8f9f99d57 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:20.239337 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff668ac3-eed7-4fa9-b470-97e3781f9a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:20.240193 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff668ac3-eed7-4fa9-b470-97e3781f9a62 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:20.240523 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff668ac3-eed7-4fa9-b470-97e3781f9a62 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:20.240801 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff668ac3-eed7-4fa9-b470-97e3781f9a62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:20.259254 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff668ac3-eed7-4fa9-b470-97e3781f9a62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-ff668ac3-eed7-4fa9-b470-97e3781f9a62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:20.267253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:20.268127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:20.268976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:20.268976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:20.268976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:20.268976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:20.268976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:20.268976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:20.268976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:20.268976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:20.268976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:20.268976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:20.268976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:20.268976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:20.268976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 188/396] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:20.274180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff668ac3-eed7-4fa9-b470-97e3781f9a62 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:20.319618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dcbfd-3c10-4744-b8d6-dbe8f9f99d57 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:20.354867 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29647d27-432a-43ce-bb4b-22d59846c913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:20.356026 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29647d27-432a-43ce-bb4b-22d59846c913 None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:20.356449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29647d27-432a-43ce-bb4b-22d59846c913 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:20.356856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29647d27-432a-43ce-bb4b-22d59846c913 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f/groups/f92a13bf1cb3480aa636cced39bb1cee/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:20.358946 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29647d27-432a-43ce-bb4b-22d59846c913 None admin] RBAC: Authorizing `identity:create_grant(project_id=36617c1516b14f15a3b4a35692ff654f, group_id=f92a13bf1cb3480aa636cced39bb1cee, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:20.360700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29647d27-432a-43ce-bb4b-22d59846c913 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:20.379070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 209/397] 10.222.0.25 () {58 vars in 1442 bytes} [Mon May 9 18:16:20 2022] PUT /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f/groups/f92a13bf1cb3480aa636cced39bb1cee/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:16:20.408700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff668ac3-eed7-4fa9-b470-97e3781f9a62 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:20.458395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d2a7b-4c81-4651-8a2e-7dd07d7d1106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:20.459006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d2a7b-4c81-4651-8a2e-7dd07d7d1106 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:20.459315 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d2a7b-4c81-4651-8a2e-7dd07d7d1106 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:20.459593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d2a7b-4c81-4651-8a2e-7dd07d7d1106 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:20.460422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d2a7b-4c81-4651-8a2e-7dd07d7d1106 None admin] RBAC: Adding query filter params (name=member) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:20.460788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d2a7b-4c81-4651-8a2e-7dd07d7d1106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:20.462196 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d2a7b-4c81-4651-8a2e-7dd07d7d1106 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:20.474380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 189/398] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:20.480066 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29647d27-432a-43ce-bb4b-22d59846c913 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:20.611815 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29647d27-432a-43ce-bb4b-22d59846c913 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:20.652836 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abae04e-6150-496a-a02c-d0515d445b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:20.654101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abae04e-6150-496a-a02c-d0515d445b04 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:20.654575 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abae04e-6150-496a-a02c-d0515d445b04 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:20.655039 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abae04e-6150-496a-a02c-d0515d445b04 None admin] PATH_INFO: `/v3/groups/f92a13bf1cb3480aa636cced39bb1cee` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:20.656545 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abae04e-6150-496a-a02c-d0515d445b04 None admin] RBAC: Authorizing `identity:get_group(group_id=f92a13bf1cb3480aa636cced39bb1cee)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:20.658196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abae04e-6150-496a-a02c-d0515d445b04 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:20.660952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 210/399] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:20 2022] GET /identity/v3/groups/f92a13bf1cb3480aa636cced39bb1cee => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:20.666160 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d2a7b-4c81-4651-8a2e-7dd07d7d1106 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:20.799100 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d2a7b-4c81-4651-8a2e-7dd07d7d1106 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:20.839412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60d2b8-74e2-482a-8bbb-3bfe6073ba18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:20.841107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60d2b8-74e2-482a-8bbb-3bfe6073ba18 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:20.841656 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60d2b8-74e2-482a-8bbb-3bfe6073ba18 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:20.842173 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60d2b8-74e2-482a-8bbb-3bfe6073ba18 None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:20.843734 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60d2b8-74e2-482a-8bbb-3bfe6073ba18 None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:20.845679 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60d2b8-74e2-482a-8bbb-3bfe6073ba18 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:20.847995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 190/400] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:20 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:20.853565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abae04e-6150-496a-a02c-d0515d445b04 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:20.960970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abae04e-6150-496a-a02c-d0515d445b04 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:20.996253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1b29a-98be-44af-b30d-c5b1ee4f31f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:20.997137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b29a-98be-44af-b30d-c5b1ee4f31f4 None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:20.997415 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b29a-98be-44af-b30d-c5b1ee4f31f4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:20.997682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b29a-98be-44af-b30d-c5b1ee4f31f4 None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835/groups/f92a13bf1cb3480aa636cced39bb1cee/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:20.999636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1b29a-98be-44af-b30d-c5b1ee4f31f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=82a98033ef0f45f887ea92601788b835, group_id=f92a13bf1cb3480aa636cced39bb1cee, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:21.001051 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1b29a-98be-44af-b30d-c5b1ee4f31f4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:21.017553 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 211/401] 10.222.0.25 () {58 vars in 1442 bytes} [Mon May 9 18:16:20 2022] PUT /identity/v3/projects/82a98033ef0f45f887ea92601788b835/groups/f92a13bf1cb3480aa636cced39bb1cee/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:16:21.487217 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bc330-fa09-4c11-8c71-2c040c5cfda2 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:21.487740 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bc330-fa09-4c11-8c71-2c040c5cfda2 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:21.488033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bc330-fa09-4c11-8c71-2c040c5cfda2 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:21.489928 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 191/402] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:21.495864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd1187-0fbe-4368-9d81-7ad144843a5b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:21.495864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd1187-0fbe-4368-9d81-7ad144843a5b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:21.495864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd1187-0fbe-4368-9d81-7ad144843a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:21.509014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ec88b-a8b9-4d8d-9888-ef9c2a98b352 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:21.509014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ec88b-a8b9-4d8d-9888-ef9c2a98b352 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:21.509014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ec88b-a8b9-4d8d-9888-ef9c2a98b352 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:21.509014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 192/403] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:16:21.514516 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05352e6-ae36-4460-b4dc-9cf69c02c33c None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:21.514912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9fdd1187-0fbe-4368-9d81-7ad144843a5b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:21.515284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05352e6-ae36-4460-b4dc-9cf69c02c33c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:21.515763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05352e6-ae36-4460-b4dc-9cf69c02c33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:21.516170 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9fdd1187-0fbe-4368-9d81-7ad144843a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:21.535530 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-e05352e6-ae36-4460-b4dc-9cf69c02c33c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:21.537484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e05352e6-ae36-4460-b4dc-9cf69c02c33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:21.614019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 193/404] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:21.626651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 212/405] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:21.637824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05352e6-ae36-4460-b4dc-9cf69c02c33c None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:21.672566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd1187-0fbe-4368-9d81-7ad144843a5b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:21.769823 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05352e6-ae36-4460-b4dc-9cf69c02c33c None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:21.808325 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbeeb72-9ef8-40b4-9e0a-af6a2a8172f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:21.810165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeeb72-9ef8-40b4-9e0a-af6a2a8172f2 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:21.810697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeeb72-9ef8-40b4-9e0a-af6a2a8172f2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:21.811431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeeb72-9ef8-40b4-9e0a-af6a2a8172f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:21.824542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbeeb72-9ef8-40b4-9e0a-af6a2a8172f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-bbbeeb72-9ef8-40b4-9e0a-af6a2a8172f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:21.835064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:21.836053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:21.836871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:21.836871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:21.836871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:21.836871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:21.836871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:21.836871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:21.836871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:21.836871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:21.836871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:21.836871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:21.836871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:16:21.836871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:21.839640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 194/406] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:16:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:21.846318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbeeb72-9ef8-40b4-9e0a-af6a2a8172f2 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:21.878753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd1187-0fbe-4368-9d81-7ad144843a5b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:21.913779 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3440a-24a7-4ce1-89ab-763c50ac0ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:21.914957 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3440a-24a7-4ce1-89ab-763c50ac0ba0 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:21.915470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3440a-24a7-4ce1-89ab-763c50ac0ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:21.915963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3440a-24a7-4ce1-89ab-763c50ac0ba0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:21.927130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3440a-24a7-4ce1-89ab-763c50ac0ba0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-b2d3440a-24a7-4ce1-89ab-763c50ac0ba0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:21.935588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:21.936453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:21.937237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:21.937237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:21.937237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:21.937237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:21.937237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:21.937237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:21.937237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:21.937237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:21.937237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:21.937237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:21.937237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:21.937237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:21.939924 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 213/407] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:16:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:21.948911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3440a-24a7-4ce1-89ab-763c50ac0ba0 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:22.027321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbeeb72-9ef8-40b4-9e0a-af6a2a8172f2 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:22.067418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43731a39-d685-4cff-b2e6-530fe359a6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:22.068430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43731a39-d685-4cff-b2e6-530fe359a6ff None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:22.068906 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43731a39-d685-4cff-b2e6-530fe359a6ff None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:22.069414 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43731a39-d685-4cff-b2e6-530fe359a6ff None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:22.070347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43731a39-d685-4cff-b2e6-530fe359a6ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:22.070842 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43731a39-d685-4cff-b2e6-530fe359a6ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:22.072426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43731a39-d685-4cff-b2e6-530fe359a6ff None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:22.081614 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 195/408] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:22.083158 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e055c4-b3ad-4498-964f-275f6f281ff6 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:22.083668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e055c4-b3ad-4498-964f-275f6f281ff6 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:22.084128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e055c4-b3ad-4498-964f-275f6f281ff6 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:22.085543 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 196/409] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:22.088540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e055c4-b3ad-4498-964f-275f6f281ff6 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:22.117552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3440a-24a7-4ce1-89ab-763c50ac0ba0 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:22.165382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065f23e-ebd2-4efb-a632-e41a9d6fef8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:22.166304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065f23e-ebd2-4efb-a632-e41a9d6fef8d None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:22.166730 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065f23e-ebd2-4efb-a632-e41a9d6fef8d None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:22.167149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065f23e-ebd2-4efb-a632-e41a9d6fef8d None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:22.168086 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065f23e-ebd2-4efb-a632-e41a9d6fef8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:22.168583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065f23e-ebd2-4efb-a632-e41a9d6fef8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:22.170637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065f23e-ebd2-4efb-a632-e41a9d6fef8d None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:22.181663 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 214/410] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:22.188258 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba931767-b6c6-4c10-9cb8-8f026f6c85bb None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:22.188706 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba931767-b6c6-4c10-9cb8-8f026f6c85bb None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:22.189286 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba931767-b6c6-4c10-9cb8-8f026f6c85bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:22.199833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e055c4-b3ad-4498-964f-275f6f281ff6 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:22.210829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-ba931767-b6c6-4c10-9cb8-8f026f6c85bb None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:22.212592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ba931767-b6c6-4c10-9cb8-8f026f6c85bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:22.236181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ffa88-bad5-4344-a9ff-c41b61f89885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:22.237706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ffa88-bad5-4344-a9ff-c41b61f89885 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:22.238222 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ffa88-bad5-4344-a9ff-c41b61f89885 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:22.238733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ffa88-bad5-4344-a9ff-c41b61f89885 None admin] PATH_INFO: `/v3/groups/aa85b36cfe21438ca82f196c6d2e6f78` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:22.240123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ffa88-bad5-4344-a9ff-c41b61f89885 None admin] RBAC: Authorizing `identity:get_group(group_id=aa85b36cfe21438ca82f196c6d2e6f78)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:22.241786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ffa88-bad5-4344-a9ff-c41b61f89885 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:22.243609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 197/411] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:22 2022] GET /identity/v3/groups/aa85b36cfe21438ca82f196c6d2e6f78 => generated 229 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:22.244578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ffa88-bad5-4344-a9ff-c41b61f89885 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:22.299355 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 215/412] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:22.301111 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba931767-b6c6-4c10-9cb8-8f026f6c85bb None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:22.368673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ffa88-bad5-4344-a9ff-c41b61f89885 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:22.408525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba931767-b6c6-4c10-9cb8-8f026f6c85bb None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:22.410023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493a708-5c7b-4f95-b39c-f44116e4142b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:22.410914 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493a708-5c7b-4f95-b39c-f44116e4142b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:22.411344 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493a708-5c7b-4f95-b39c-f44116e4142b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:22.411621 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493a708-5c7b-4f95-b39c-f44116e4142b None admin] PATH_INFO: `/v3/groups/f92a13bf1cb3480aa636cced39bb1cee` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:22.413583 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493a708-5c7b-4f95-b39c-f44116e4142b None admin] RBAC: Authorizing `identity:get_group(group_id=f92a13bf1cb3480aa636cced39bb1cee)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:22.415002 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493a708-5c7b-4f95-b39c-f44116e4142b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:22.416665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 198/413] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:22 2022] GET /identity/v3/groups/f92a13bf1cb3480aa636cced39bb1cee => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:22.417494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493a708-5c7b-4f95-b39c-f44116e4142b None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:22.451960 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebbca1-dcbe-435e-b641-1b8edeb63d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:22.452955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbca1-dcbe-435e-b641-1b8edeb63d12 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:22.453252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbca1-dcbe-435e-b641-1b8edeb63d12 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:22.453529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbca1-dcbe-435e-b641-1b8edeb63d12 None admin] PATH_INFO: `/v3/projects/969d4448da94407dbcabadb61787000d` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:22.455280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebbca1-dcbe-435e-b641-1b8edeb63d12 None admin] RBAC: Authorizing `identity:get_project(project_id=969d4448da94407dbcabadb61787000d)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:22.456747 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebbca1-dcbe-435e-b641-1b8edeb63d12 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:22.459314 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 216/414] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:22 2022] GET /identity/v3/projects/969d4448da94407dbcabadb61787000d => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:22.460785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebbca1-dcbe-435e-b641-1b8edeb63d12 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:22.537256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493a708-5c7b-4f95-b39c-f44116e4142b None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:22.572248 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebbca1-dcbe-435e-b641-1b8edeb63d12 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:22.582678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01128dea-ea24-4e17-84f9-c3092ac90597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:22.583889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01128dea-ea24-4e17-84f9-c3092ac90597 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:22.584179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01128dea-ea24-4e17-84f9-c3092ac90597 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:22.584461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01128dea-ea24-4e17-84f9-c3092ac90597 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:22.600252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01128dea-ea24-4e17-84f9-c3092ac90597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:22.608905 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d9fb4-49f2-4b36-be32-e9a962b92181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:22.610019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d9fb4-49f2-4b36-be32-e9a962b92181 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:22.610490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d9fb4-49f2-4b36-be32-e9a962b92181 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-01128dea-ea24-4e17-84f9-c3092ac90597 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:22.610849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:22.611746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:22.613194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:22.613194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:22.613194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:22.613194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:22.613194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:22.613194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:22.613194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:22.613194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:22.613194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:22.613194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:22.613194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:22.613194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:22.614530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d9fb4-49f2-4b36-be32-e9a962b92181 None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:22.617613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 199/415] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:16:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:22.618434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01128dea-ea24-4e17-84f9-c3092ac90597 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:22.618537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d9fb4-49f2-4b36-be32-e9a962b92181 None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:22.620266 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d9fb4-49f2-4b36-be32-e9a962b92181 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:22.622164 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 217/416] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:22 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:22.623096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d9fb4-49f2-4b36-be32-e9a962b92181 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:22.719870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01128dea-ea24-4e17-84f9-c3092ac90597 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:22.751109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d9fb4-49f2-4b36-be32-e9a962b92181 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:22.756255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9598c4-3b56-4be4-8a2a-d008f031e561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:22.757381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9598c4-3b56-4be4-8a2a-d008f031e561 None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:22.757856 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9598c4-3b56-4be4-8a2a-d008f031e561 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:22.758321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9598c4-3b56-4be4-8a2a-d008f031e561 None admin] PATH_INFO: `/v3/projects/969d4448da94407dbcabadb61787000d/groups/aa85b36cfe21438ca82f196c6d2e6f78/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:22.761101 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9598c4-3b56-4be4-8a2a-d008f031e561 None admin] RBAC: Authorizing `identity:create_grant(project_id=969d4448da94407dbcabadb61787000d, group_id=aa85b36cfe21438ca82f196c6d2e6f78, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:22.762746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9598c4-3b56-4be4-8a2a-d008f031e561 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:22.777826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 200/417] 10.222.0.25 () {58 vars in 1442 bytes} [Mon May 9 18:16:22 2022] PUT /identity/v3/projects/969d4448da94407dbcabadb61787000d/groups/aa85b36cfe21438ca82f196c6d2e6f78/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:16:22.778748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9598c4-3b56-4be4-8a2a-d008f031e561 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:22.794208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a655c-19f9-45ad-b136-179bc46a563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:22.795633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a655c-19f9-45ad-b136-179bc46a563a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:22.796456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a655c-19f9-45ad-b136-179bc46a563a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:22.796928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a655c-19f9-45ad-b136-179bc46a563a None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:22.798165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a655c-19f9-45ad-b136-179bc46a563a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:22.799080 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a655c-19f9-45ad-b136-179bc46a563a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:22.801215 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a655c-19f9-45ad-b136-179bc46a563a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:22.821622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 218/418] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:22.829443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a655c-19f9-45ad-b136-179bc46a563a None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:22.939859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a655c-19f9-45ad-b136-179bc46a563a None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:22.958010 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9598c4-3b56-4be4-8a2a-d008f031e561 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:22.976010 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9431d-6e6b-4e18-9865-aa45f952d4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:22.976680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9431d-6e6b-4e18-9865-aa45f952d4d9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:22.976962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9431d-6e6b-4e18-9865-aa45f952d4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:22.977240 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9431d-6e6b-4e18-9865-aa45f952d4d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:23.031280 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff6f36-709b-4324-bd7e-641d901a021f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:23.033402 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff6f36-709b-4324-bd7e-641d901a021f None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:23.033881 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff6f36-709b-4324-bd7e-641d901a021f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:23.035070 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff6f36-709b-4324-bd7e-641d901a021f None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835/groups/f92a13bf1cb3480aa636cced39bb1cee/roles/9d45ca30c3584deabd69f2cd196ae5cc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:23.037878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff6f36-709b-4324-bd7e-641d901a021f None admin] RBAC: Authorizing `identity:create_grant(project_id=82a98033ef0f45f887ea92601788b835, group_id=f92a13bf1cb3480aa636cced39bb1cee, role_id=9d45ca30c3584deabd69f2cd196ae5cc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:23.040059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff6f36-709b-4324-bd7e-641d901a021f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:23.057001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9431d-6e6b-4e18-9865-aa45f952d4d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:23.057743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 201/419] 10.222.0.25 () {58 vars in 1442 bytes} [Mon May 9 18:16:22 2022] PUT /identity/v3/projects/82a98033ef0f45f887ea92601788b835/groups/f92a13bf1cb3480aa636cced39bb1cee/roles/9d45ca30c3584deabd69f2cd196ae5cc => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:16:23.058719 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9431d-6e6b-4e18-9865-aa45f952d4d9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:23.169519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 219/420] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 18:16:23.175495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff6f36-709b-4324-bd7e-641d901a021f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:23.281068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72287d83-cd20-44f0-8894-dbbb679386bf None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:23.281388 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72287d83-cd20-44f0-8894-dbbb679386bf None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:23.281629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72287d83-cd20-44f0-8894-dbbb679386bf None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:23.285704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 220/421] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:23.290569 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd5ff0-1fdd-4ee2-9c26-fa373cf70b22 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:23.290854 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd5ff0-1fdd-4ee2-9c26-fa373cf70b22 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:23.291333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd5ff0-1fdd-4ee2-9c26-fa373cf70b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:23.302486 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff6f36-709b-4324-bd7e-641d901a021f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:23.314436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-dcbd5ff0-1fdd-4ee2-9c26-fa373cf70b22 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:23.315776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-dcbd5ff0-1fdd-4ee2-9c26-fa373cf70b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:23.343697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca79677-15c8-41f9-b37a-c47704348ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:23.344828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca79677-15c8-41f9-b37a-c47704348ab9 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:23.345114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca79677-15c8-41f9-b37a-c47704348ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:23.345552 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca79677-15c8-41f9-b37a-c47704348ab9 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:23.347892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca79677-15c8-41f9-b37a-c47704348ab9 None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:23.349746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca79677-15c8-41f9-b37a-c47704348ab9 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:23.355268 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 202/422] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:23 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:23.359659 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca79677-15c8-41f9-b37a-c47704348ab9 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:23.399323 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 221/423] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:23.421089 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd5ff0-1fdd-4ee2-9c26-fa373cf70b22 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:23.551250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca79677-15c8-41f9-b37a-c47704348ab9 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:23.602701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd5ff0-1fdd-4ee2-9c26-fa373cf70b22 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:23.624810 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b5151-b2da-47b7-8695-96c8d2b2155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:23.625680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b5151-b2da-47b7-8695-96c8d2b2155d None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:23.625964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b5151-b2da-47b7-8695-96c8d2b2155d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:23.626284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b5151-b2da-47b7-8695-96c8d2b2155d None admin] PATH_INFO: `/v3/groups/f92a13bf1cb3480aa636cced39bb1cee` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:23.631859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b5151-b2da-47b7-8695-96c8d2b2155d None admin] RBAC: Authorizing `identity:get_group(group_id=f92a13bf1cb3480aa636cced39bb1cee)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:23.633333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b5151-b2da-47b7-8695-96c8d2b2155d None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:23.635301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 203/424] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:23 2022] GET /identity/v3/groups/f92a13bf1cb3480aa636cced39bb1cee => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:23.640972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b5151-b2da-47b7-8695-96c8d2b2155d None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:23.652760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083983d-8dde-4f24-b409-56e970a89f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:23.653573 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083983d-8dde-4f24-b409-56e970a89f42 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:23.653854 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083983d-8dde-4f24-b409-56e970a89f42 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:23.654128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083983d-8dde-4f24-b409-56e970a89f42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:23.673687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083983d-8dde-4f24-b409-56e970a89f42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-c083983d-8dde-4f24-b409-56e970a89f42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:23.681463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:23.682348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:23.683147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:23.683147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:23.683147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:23.683147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:23.683147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:23.683147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:23.683147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:23.683147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:23.683147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:23.683147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:23.683147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:23.683147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:23.683147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 222/425] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:23.696196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083983d-8dde-4f24-b409-56e970a89f42 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:23.770925 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b5151-b2da-47b7-8695-96c8d2b2155d None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:23.805839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083983d-8dde-4f24-b409-56e970a89f42 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:23.813821 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b922802-cf0a-4baa-ab30-9a47d7b3abaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:23.814509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922802-cf0a-4baa-ab30-9a47d7b3abaa None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:23.814783 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922802-cf0a-4baa-ab30-9a47d7b3abaa None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:23.815068 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922802-cf0a-4baa-ab30-9a47d7b3abaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:23.816473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b922802-cf0a-4baa-ab30-9a47d7b3abaa None admin] RBAC: Adding query filter params (group.id=f92a13bf1cb3480aa636cced39bb1cee, role.id=9d45ca30c3584deabd69f2cd196ae5cc, scope.project.id=36617c1516b14f15a3b4a35692ff654f) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:23.816784 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b922802-cf0a-4baa-ab30-9a47d7b3abaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:23.818210 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b922802-cf0a-4baa-ab30-9a47d7b3abaa None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:23.831574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 204/426] 10.222.0.25 () {56 vars in 1376 bytes} [Mon May 9 18:16:23 2022] GET /identity/v3/role_assignments?group.id=f92a13bf1cb3480aa636cced39bb1cee&scope.project.id=36617c1516b14f15a3b4a35692ff654f&role.id=9d45ca30c3584deabd69f2cd196ae5cc => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:23.849689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264518dc-3692-4922-b791-8d67f1fa56db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:23.850395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264518dc-3692-4922-b791-8d67f1fa56db None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:23.850730 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264518dc-3692-4922-b791-8d67f1fa56db None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:23.851015 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264518dc-3692-4922-b791-8d67f1fa56db None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:23.851889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264518dc-3692-4922-b791-8d67f1fa56db None admin] RBAC: Adding query filter params (name=member) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:23.852198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264518dc-3692-4922-b791-8d67f1fa56db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:23.853685 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264518dc-3692-4922-b791-8d67f1fa56db None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:23.873034 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 223/427] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:23.879440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b922802-cf0a-4baa-ab30-9a47d7b3abaa None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:23.929744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82756e07-f55b-4561-9bc6-ef0489b5ccf2 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:23.930038 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82756e07-f55b-4561-9bc6-ef0489b5ccf2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:23.930204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82756e07-f55b-4561-9bc6-ef0489b5ccf2 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:23.931828 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 224/428] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:16:23.948762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b28c34-a00f-4906-98d7-52c4154d93c4 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:23.949104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b28c34-a00f-4906-98d7-52c4154d93c4 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:23.949325 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b28c34-a00f-4906-98d7-52c4154d93c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:23.984040 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-49b28c34-a00f-4906-98d7-52c4154d93c4 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:23.986670 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-49b28c34-a00f-4906-98d7-52c4154d93c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:24.049950 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b922802-cf0a-4baa-ab30-9a47d7b3abaa None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:24.093863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 225/429] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:24.095983 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c0b9b-9800-4ffc-8661-d851f50355ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:24.096728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c0b9b-9800-4ffc-8661-d851f50355ce None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:24.097014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c0b9b-9800-4ffc-8661-d851f50355ce None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:24.097293 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c0b9b-9800-4ffc-8661-d851f50355ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:24.121883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b28c34-a00f-4906-98d7-52c4154d93c4 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:24.174573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c0b9b-9800-4ffc-8661-d851f50355ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:24.177585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c0b9b-9800-4ffc-8661-d851f50355ce None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:24.235043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b28c34-a00f-4906-98d7-52c4154d93c4 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:24.270849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552192d-b140-42d8-9211-bc49abe8d114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:24.272130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552192d-b140-42d8-9211-bc49abe8d114 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:24.272620 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552192d-b140-42d8-9211-bc49abe8d114 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:24.273121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552192d-b140-42d8-9211-bc49abe8d114 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:24.282609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 205/430] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:24.287406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552192d-b140-42d8-9211-bc49abe8d114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:24.289681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c0b9b-9800-4ffc-8661-d851f50355ce None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-0552192d-b140-42d8-9211-bc49abe8d114 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:24.296053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:24.296922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:24.297737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:24.297737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:24.297737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:24.297737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:24.297737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:24.297737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:24.297737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:24.297737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:24.297737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:24.297737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:24.297737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 18:16:24.297737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:24.300462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 226/431] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:24.306582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552192d-b140-42d8-9211-bc49abe8d114 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:24.402294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c0b9b-9800-4ffc-8661-d851f50355ce None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:24.438630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552192d-b140-42d8-9211-bc49abe8d114 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:24.443529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cd693-89b2-4dfd-94fc-a013b544c535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:24.444627 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cd693-89b2-4dfd-94fc-a013b544c535 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:24.445102 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cd693-89b2-4dfd-94fc-a013b544c535 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:24.445977 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cd693-89b2-4dfd-94fc-a013b544c535 None admin] PATH_INFO: `/v3/projects/36617c1516b14f15a3b4a35692ff654f` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:24.447744 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338cd693-89b2-4dfd-94fc-a013b544c535 None admin] RBAC: Authorizing `identity:get_project(project_id=36617c1516b14f15a3b4a35692ff654f)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:24.449973 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338cd693-89b2-4dfd-94fc-a013b544c535 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:24.457504 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 206/432] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:24 2022] GET /identity/v3/projects/36617c1516b14f15a3b4a35692ff654f => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:24.458490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cd693-89b2-4dfd-94fc-a013b544c535 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:24.482649 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10782814-4b02-4d1b-8c5d-1a59491e943d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:24.483406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782814-4b02-4d1b-8c5d-1a59491e943d None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:24.483690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782814-4b02-4d1b-8c5d-1a59491e943d None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:24.484017 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782814-4b02-4d1b-8c5d-1a59491e943d None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:24.484926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10782814-4b02-4d1b-8c5d-1a59491e943d None admin] RBAC: Adding query filter params (name=member) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:24.485142 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10782814-4b02-4d1b-8c5d-1a59491e943d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:24.486467 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10782814-4b02-4d1b-8c5d-1a59491e943d None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:24.495558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 227/433] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:16:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:24.500456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10782814-4b02-4d1b-8c5d-1a59491e943d None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:24.584749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cd693-89b2-4dfd-94fc-a013b544c535 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:24.620705 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10782814-4b02-4d1b-8c5d-1a59491e943d None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:24.633692 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa29cae-0a13-4f1c-b245-e13f44969ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:24.634547 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa29cae-0a13-4f1c-b245-e13f44969ea5 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:24.634822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa29cae-0a13-4f1c-b245-e13f44969ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:24.635091 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa29cae-0a13-4f1c-b245-e13f44969ea5 None admin] PATH_INFO: `/v3/groups/f92a13bf1cb3480aa636cced39bb1cee` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:24.636839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa29cae-0a13-4f1c-b245-e13f44969ea5 None admin] RBAC: Authorizing `identity:get_group(group_id=f92a13bf1cb3480aa636cced39bb1cee)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:24.638253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa29cae-0a13-4f1c-b245-e13f44969ea5 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:24.645168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 207/434] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:24 2022] GET /identity/v3/groups/f92a13bf1cb3480aa636cced39bb1cee => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:24.646181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa29cae-0a13-4f1c-b245-e13f44969ea5 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:24.667416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0dac98-4720-48e4-ae15-fb8ed586734f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:24.668116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dac98-4720-48e4-ae15-fb8ed586734f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:24.668396 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dac98-4720-48e4-ae15-fb8ed586734f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:24.668717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dac98-4720-48e4-ae15-fb8ed586734f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:24.733842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0dac98-4720-48e4-ae15-fb8ed586734f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:24.735433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0dac98-4720-48e4-ae15-fb8ed586734f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:24.772586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa29cae-0a13-4f1c-b245-e13f44969ea5 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:24.815854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16960b-77a9-4bf7-bee3-dd80c415406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:24.816857 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16960b-77a9-4bf7-bee3-dd80c415406f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:24.817331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16960b-77a9-4bf7-bee3-dd80c415406f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:24.817797 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16960b-77a9-4bf7-bee3-dd80c415406f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:24.819250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16960b-77a9-4bf7-bee3-dd80c415406f None admin] RBAC: Adding query filter params (group.id=f92a13bf1cb3480aa636cced39bb1cee, role.id=e1dddf0b88ee4578822ce51a2aba6760, scope.project.id=36617c1516b14f15a3b4a35692ff654f) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:24.820034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16960b-77a9-4bf7-bee3-dd80c415406f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:24.822115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16960b-77a9-4bf7-bee3-dd80c415406f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:24.835443 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 208/436] 10.222.0.25 () {56 vars in 1376 bytes} [Mon May 9 18:16:24 2022] GET /identity/v3/role_assignments?group.id=f92a13bf1cb3480aa636cced39bb1cee&scope.project.id=36617c1516b14f15a3b4a35692ff654f&role.id=e1dddf0b88ee4578822ce51a2aba6760 => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:24.835618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 228/436] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 18:16:24.836625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0dac98-4720-48e4-ae15-fb8ed586734f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:24.942711 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0dac98-4720-48e4-ae15-fb8ed586734f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:24.979370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624751a3-084a-4e77-b29d-c0250a2cd1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:24.980238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624751a3-084a-4e77-b29d-c0250a2cd1e3 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:24.980792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624751a3-084a-4e77-b29d-c0250a2cd1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:24.981066 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624751a3-084a-4e77-b29d-c0250a2cd1e3 None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:24.982275 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624751a3-084a-4e77-b29d-c0250a2cd1e3 None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:24.983925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624751a3-084a-4e77-b29d-c0250a2cd1e3 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:24.986096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 229/437] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:24 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:24.992349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16960b-77a9-4bf7-bee3-dd80c415406f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:25.098459 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16960b-77a9-4bf7-bee3-dd80c415406f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:25.132208 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f481f18-a98f-4cc9-947d-0832d2111aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:25.133418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f481f18-a98f-4cc9-947d-0832d2111aa2 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:25.133866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f481f18-a98f-4cc9-947d-0832d2111aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:25.134141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f481f18-a98f-4cc9-947d-0832d2111aa2 None admin] PATH_INFO: `/v3/groups/f92a13bf1cb3480aa636cced39bb1cee` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:25.135336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f481f18-a98f-4cc9-947d-0832d2111aa2 None admin] RBAC: Authorizing `identity:get_group(group_id=f92a13bf1cb3480aa636cced39bb1cee)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:25.136745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f481f18-a98f-4cc9-947d-0832d2111aa2 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:25.138528 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 209/438] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:24 2022] GET /identity/v3/groups/f92a13bf1cb3480aa636cced39bb1cee => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:25.144687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624751a3-084a-4e77-b29d-c0250a2cd1e3 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:25.249619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624751a3-084a-4e77-b29d-c0250a2cd1e3 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:25.286391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9749bf-83ec-4d70-a998-c87d8a0c5960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:25.287400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9749bf-83ec-4d70-a998-c87d8a0c5960 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:25.287893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9749bf-83ec-4d70-a998-c87d8a0c5960 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:25.288358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9749bf-83ec-4d70-a998-c87d8a0c5960 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:25.289381 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9749bf-83ec-4d70-a998-c87d8a0c5960 None admin] RBAC: Adding query filter params (group.id=f92a13bf1cb3480aa636cced39bb1cee, role.id=e1dddf0b88ee4578822ce51a2aba6760, scope.project.id=82a98033ef0f45f887ea92601788b835) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:25.289899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9749bf-83ec-4d70-a998-c87d8a0c5960 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:25.291515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9749bf-83ec-4d70-a998-c87d8a0c5960 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:25.298224 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 230/439] 10.222.0.25 () {56 vars in 1376 bytes} [Mon May 9 18:16:25 2022] GET /identity/v3/role_assignments?group.id=f92a13bf1cb3480aa636cced39bb1cee&scope.project.id=82a98033ef0f45f887ea92601788b835&role.id=e1dddf0b88ee4578822ce51a2aba6760 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:25.399379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad910f-df8e-4603-8bdd-53e9a4601c46 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:25.399979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad910f-df8e-4603-8bdd-53e9a4601c46 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:25.400261 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad910f-df8e-4603-8bdd-53e9a4601c46 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:25.401966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 210/440] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:25.408568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de2ec6-4b8b-46d1-ac3d-a5c7b4bb460f None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:25.409247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de2ec6-4b8b-46d1-ac3d-a5c7b4bb460f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:25.409759 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de2ec6-4b8b-46d1-ac3d-a5c7b4bb460f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:25.429623 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-43de2ec6-4b8b-46d1-ac3d-a5c7b4bb460f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:25.431227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-43de2ec6-4b8b-46d1-ac3d-a5c7b4bb460f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:25.512192 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 231/441] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:25.535966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad910f-df8e-4603-8bdd-53e9a4601c46 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:25.598887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee0125-efe6-48c4-ac88-690dbe7f3bfb None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:25.599258 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee0125-efe6-48c4-ac88-690dbe7f3bfb None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:25.599502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee0125-efe6-48c4-ac88-690dbe7f3bfb None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:25.602728 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 232/442] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:25.608260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1da7b5-88ec-4f40-bc5a-b9a10c27a564 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:25.608803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1da7b5-88ec-4f40-bc5a-b9a10c27a564 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:25.609083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1da7b5-88ec-4f40-bc5a-b9a10c27a564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:25.626548 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-dc1da7b5-88ec-4f40-bc5a-b9a10c27a564 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:25.630214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-dc1da7b5-88ec-4f40-bc5a-b9a10c27a564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:25.646878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad910f-df8e-4603-8bdd-53e9a4601c46 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:25.683306 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c8a80-4f26-44fa-9df7-17d34c888db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:25.684115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c8a80-4f26-44fa-9df7-17d34c888db6 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:25.684396 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c8a80-4f26-44fa-9df7-17d34c888db6 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:25.684671 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c8a80-4f26-44fa-9df7-17d34c888db6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:25.697905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c8a80-4f26-44fa-9df7-17d34c888db6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:25.703216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 233/443] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-115c8a80-4f26-44fa-9df7-17d34c888db6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:25.705622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:25.706540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:25.707376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:25.707376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:25.707376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:25.707376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:25.707376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:25.707376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:25.707376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:25.707376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:25.707376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:25.707376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:25.707376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:16:25.707376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:25.707376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 211/444] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:16:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:25.712505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1da7b5-88ec-4f40-bc5a-b9a10c27a564 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:25.725875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c8a80-4f26-44fa-9df7-17d34c888db6 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:25.811908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1da7b5-88ec-4f40-bc5a-b9a10c27a564 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:25.820757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c8a80-4f26-44fa-9df7-17d34c888db6 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:25.843425 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727873c-530e-471e-9bd8-6433db660596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:25.844087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727873c-530e-471e-9bd8-6433db660596 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:25.844364 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727873c-530e-471e-9bd8-6433db660596 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:25.844635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727873c-530e-471e-9bd8-6433db660596 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:25.845367 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727873c-530e-471e-9bd8-6433db660596 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:25.845667 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727873c-530e-471e-9bd8-6433db660596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:25.847039 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727873c-530e-471e-9bd8-6433db660596 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:25.852010 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d1388-c3de-40c0-8620-ead4d5ff77af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:25.852820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d1388-c3de-40c0-8620-ead4d5ff77af None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:25.853103 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d1388-c3de-40c0-8620-ead4d5ff77af None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:25.853377 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d1388-c3de-40c0-8620-ead4d5ff77af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:25.854596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 234/445] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:25.860047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727873c-530e-471e-9bd8-6433db660596 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:25.864353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d1388-c3de-40c0-8620-ead4d5ff77af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-a48d1388-c3de-40c0-8620-ead4d5ff77af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:25.872395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:25.873250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:25.874041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:25.874041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:25.874041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:25.874041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:25.874041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:25.874041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:25.874041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:25.874041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:25.874041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:25.874041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:25.874041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:25.874041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:25.874041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 212/446] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:16:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:25.879869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d1388-c3de-40c0-8620-ead4d5ff77af None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:25.955973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727873c-530e-471e-9bd8-6433db660596 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:25.973419 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d1388-c3de-40c0-8620-ead4d5ff77af None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:25.991873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240b0cf-d8b7-4f5f-9d58-82eac1fc9e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:25.992518 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240b0cf-d8b7-4f5f-9d58-82eac1fc9e84 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:25.992801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240b0cf-d8b7-4f5f-9d58-82eac1fc9e84 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:25.993079 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240b0cf-d8b7-4f5f-9d58-82eac1fc9e84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:26.005593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd03393-3fc8-4303-b7de-18d1745c2343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:26.006251 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd03393-3fc8-4303-b7de-18d1745c2343 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:26.006530 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd03393-3fc8-4303-b7de-18d1745c2343 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:26.006803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd03393-3fc8-4303-b7de-18d1745c2343 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:26.007570 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd03393-3fc8-4303-b7de-18d1745c2343 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:26.007873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd03393-3fc8-4303-b7de-18d1745c2343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:26.009224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd03393-3fc8-4303-b7de-18d1745c2343 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:26.016796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 213/447] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:26.022245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd03393-3fc8-4303-b7de-18d1745c2343 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:26.056690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240b0cf-d8b7-4f5f-9d58-82eac1fc9e84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:26.058151 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240b0cf-d8b7-4f5f-9d58-82eac1fc9e84 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:26.117993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd03393-3fc8-4303-b7de-18d1745c2343 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:26.149288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5670fd2-e5b0-4536-8f71-8b018761defa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:26.149936 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5670fd2-e5b0-4536-8f71-8b018761defa None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:26.150221 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5670fd2-e5b0-4536-8f71-8b018761defa None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:26.150492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5670fd2-e5b0-4536-8f71-8b018761defa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:26.155810 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 235/448] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 18:16:26.161953 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240b0cf-d8b7-4f5f-9d58-82eac1fc9e84 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:26.213215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5670fd2-e5b0-4536-8f71-8b018761defa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:26.214643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5670fd2-e5b0-4536-8f71-8b018761defa None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:26.253130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240b0cf-d8b7-4f5f-9d58-82eac1fc9e84 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:26.285188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc537fd-3eef-45a1-810b-0243c7833e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:26.286044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc537fd-3eef-45a1-810b-0243c7833e77 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:26.286322 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc537fd-3eef-45a1-810b-0243c7833e77 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:26.286588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc537fd-3eef-45a1-810b-0243c7833e77 None admin] PATH_INFO: `/v3/projects/969d4448da94407dbcabadb61787000d` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:26.287770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc537fd-3eef-45a1-810b-0243c7833e77 None admin] RBAC: Authorizing `identity:get_project(project_id=969d4448da94407dbcabadb61787000d)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:26.289202 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc537fd-3eef-45a1-810b-0243c7833e77 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:26.291037 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 236/449] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:26 2022] GET /identity/v3/projects/969d4448da94407dbcabadb61787000d => generated 344 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:26.296403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc537fd-3eef-45a1-810b-0243c7833e77 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:26.307741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 214/450] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 18:16:26.313120 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5670fd2-e5b0-4536-8f71-8b018761defa None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:26.387906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc537fd-3eef-45a1-810b-0243c7833e77 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:26.402874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5670fd2-e5b0-4536-8f71-8b018761defa None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:26.419486 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e9c9f-d5b4-40af-8dff-a480063e591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:26.420325 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e9c9f-d5b4-40af-8dff-a480063e591c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:26.420599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e9c9f-d5b4-40af-8dff-a480063e591c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:26.420864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e9c9f-d5b4-40af-8dff-a480063e591c None admin] PATH_INFO: `/v3/groups/aa85b36cfe21438ca82f196c6d2e6f78` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:26.421998 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e9c9f-d5b4-40af-8dff-a480063e591c None admin] RBAC: Authorizing `identity:get_group(group_id=aa85b36cfe21438ca82f196c6d2e6f78)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:26.423434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e9c9f-d5b4-40af-8dff-a480063e591c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:26.425622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 237/451] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:26 2022] GET /identity/v3/groups/aa85b36cfe21438ca82f196c6d2e6f78 => generated 229 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:26.431385 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e9c9f-d5b4-40af-8dff-a480063e591c None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:26.435817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e64c4d-a956-4f2d-8e0c-c8277bfb0f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:26.436694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e64c4d-a956-4f2d-8e0c-c8277bfb0f54 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:26.436979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e64c4d-a956-4f2d-8e0c-c8277bfb0f54 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:26.437254 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e64c4d-a956-4f2d-8e0c-c8277bfb0f54 None admin] PATH_INFO: `/v3/projects/82a98033ef0f45f887ea92601788b835` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:26.438417 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e64c4d-a956-4f2d-8e0c-c8277bfb0f54 None admin] RBAC: Authorizing `identity:get_project(project_id=82a98033ef0f45f887ea92601788b835)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:26.439847 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e64c4d-a956-4f2d-8e0c-c8277bfb0f54 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:26.441510 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 215/452] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:16:26 2022] GET /identity/v3/projects/82a98033ef0f45f887ea92601788b835 => generated 302 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:26.446639 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e64c4d-a956-4f2d-8e0c-c8277bfb0f54 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:26.522093 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e9c9f-d5b4-40af-8dff-a480063e591c None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:26.538578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e64c4d-a956-4f2d-8e0c-c8277bfb0f54 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:26.553983 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e061a-d3a4-4561-b2e5-634cd17027a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:26.554637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e061a-d3a4-4561-b2e5-634cd17027a4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:26.554920 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e061a-d3a4-4561-b2e5-634cd17027a4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:26.555216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e061a-d3a4-4561-b2e5-634cd17027a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:26.556042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e061a-d3a4-4561-b2e5-634cd17027a4 None admin] RBAC: Adding query filter params (group.id=aa85b36cfe21438ca82f196c6d2e6f78, role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=969d4448da94407dbcabadb61787000d) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:26.556356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e061a-d3a4-4561-b2e5-634cd17027a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:26.557768 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e061a-d3a4-4561-b2e5-634cd17027a4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:26.563367 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 238/453] 10.222.0.25 () {56 vars in 1376 bytes} [Mon May 9 18:16:26 2022] GET /identity/v3/role_assignments?group.id=aa85b36cfe21438ca82f196c6d2e6f78&scope.project.id=969d4448da94407dbcabadb61787000d&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 611 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:26.570532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d1642-16fc-4319-8e88-b73d5d9326b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:26.571451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d1642-16fc-4319-8e88-b73d5d9326b9 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:26.571735 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d1642-16fc-4319-8e88-b73d5d9326b9 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:26.572012 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d1642-16fc-4319-8e88-b73d5d9326b9 None admin] PATH_INFO: `/v3/groups/f92a13bf1cb3480aa636cced39bb1cee` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:26.573259 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d1642-16fc-4319-8e88-b73d5d9326b9 None admin] RBAC: Authorizing `identity:get_group(group_id=f92a13bf1cb3480aa636cced39bb1cee)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:26.574723 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d1642-16fc-4319-8e88-b73d5d9326b9 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:26.576452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 216/454] 10.222.0.25 () {56 vars in 1180 bytes} [Mon May 9 18:16:26 2022] GET /identity/v3/groups/f92a13bf1cb3480aa636cced39bb1cee => generated 226 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:26.583035 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e061a-d3a4-4561-b2e5-634cd17027a4 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:26.683790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e061a-d3a4-4561-b2e5-634cd17027a4 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:26.717828 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3a350-1bcf-41ab-ae5d-d21242f5f586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:26.718494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a350-1bcf-41ab-ae5d-d21242f5f586 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:26.718777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a350-1bcf-41ab-ae5d-d21242f5f586 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:26.719056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a350-1bcf-41ab-ae5d-d21242f5f586 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:26.719919 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3a350-1bcf-41ab-ae5d-d21242f5f586 None admin] RBAC: Adding query filter params (group.id=f92a13bf1cb3480aa636cced39bb1cee, role.id=9d45ca30c3584deabd69f2cd196ae5cc, scope.project.id=82a98033ef0f45f887ea92601788b835) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:26.720236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3a350-1bcf-41ab-ae5d-d21242f5f586 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:26.722259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3a350-1bcf-41ab-ae5d-d21242f5f586 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:26.728615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 239/455] 10.222.0.25 () {56 vars in 1376 bytes} [Mon May 9 18:16:26 2022] GET /identity/v3/role_assignments?group.id=f92a13bf1cb3480aa636cced39bb1cee&scope.project.id=82a98033ef0f45f887ea92601788b835&role.id=9d45ca30c3584deabd69f2cd196ae5cc => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:29.662086 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6b7b6-5276-49a0-94b0-1edb41e7baf9 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:29.663121 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6b7b6-5276-49a0-94b0-1edb41e7baf9 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:29.663646 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6b7b6-5276-49a0-94b0-1edb41e7baf9 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:29.665320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 217/456] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:29.670744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2af61-07af-4ad5-bb5f-69776d1b1f3e None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:29.671583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2af61-07af-4ad5-bb5f-69776d1b1f3e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:29.672060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2af61-07af-4ad5-bb5f-69776d1b1f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:29.692761 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-78e2af61-07af-4ad5-bb5f-69776d1b1f3e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:29.694307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-78e2af61-07af-4ad5-bb5f-69776d1b1f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:29.782238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f51a9-c88f-4778-8a65-a3effc77f318 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:29.782870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f51a9-c88f-4778-8a65-a3effc77f318 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:29.783374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f51a9-c88f-4778-8a65-a3effc77f318 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:29.785740 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 218/457] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:29.790196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 240/458] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:29.791589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b6d87-bd8d-4dd0-ada0-c0c4d00bf42d None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:29.792073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b6d87-bd8d-4dd0-ada0-c0c4d00bf42d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:29.792526 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b6d87-bd8d-4dd0-ada0-c0c4d00bf42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:29.814561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2af61-07af-4ad5-bb5f-69776d1b1f3e None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:29.828169 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-4f6b6d87-bd8d-4dd0-ada0-c0c4d00bf42d None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:29.829362 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-4f6b6d87-bd8d-4dd0-ada0-c0c4d00bf42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:29.953249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 219/459] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:29.973892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2af61-07af-4ad5-bb5f-69776d1b1f3e None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:29.991284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea58e6-299b-4381-8d5d-d0e80a1e1eb9 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:29.991595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea58e6-299b-4381-8d5d-d0e80a1e1eb9 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:29.991849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea58e6-299b-4381-8d5d-d0e80a1e1eb9 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:29.995845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 220/460] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:16:29.998130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea58e6-299b-4381-8d5d-d0e80a1e1eb9 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:30.016026 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c58110-bbd8-4a4f-9877-a03b2701bac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:30.016839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c58110-bbd8-4a4f-9877-a03b2701bac4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:30.017120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c58110-bbd8-4a4f-9877-a03b2701bac4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:30.017391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c58110-bbd8-4a4f-9877-a03b2701bac4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:30.030316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c58110-bbd8-4a4f-9877-a03b2701bac4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:30.033013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c58110-bbd8-4a4f-9877-a03b2701bac4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-35c58110-bbd8-4a4f-9877-a03b2701bac4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:30.055464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:30.056348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:30.057630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:30.057630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:30.057630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:30.057630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:30.057630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 241/461] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:30.058209 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcb2dd-5b61-4dec-8acf-a1e13acb7702 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:30.058496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcb2dd-5b61-4dec-8acf-a1e13acb7702 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:30.063544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcb2dd-5b61-4dec-8acf-a1e13acb7702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:30.080347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-3cbcb2dd-5b61-4dec-8acf-a1e13acb7702 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:30.082052 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3cbcb2dd-5b61-4dec-8acf-a1e13acb7702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:30.127936 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea58e6-299b-4381-8d5d-d0e80a1e1eb9 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:30.156373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 242/462] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:30.157454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbcb2dd-5b61-4dec-8acf-a1e13acb7702 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:30.191727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365b6c2-873d-49c5-925f-0683765661c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:30.192528 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365b6c2-873d-49c5-925f-0683765661c7 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:30.193126 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365b6c2-873d-49c5-925f-0683765661c7 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:30.193571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365b6c2-873d-49c5-925f-0683765661c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:30.207015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365b6c2-873d-49c5-925f-0683765661c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:30.209112 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365b6c2-873d-49c5-925f-0683765661c7 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-c365b6c2-873d-49c5-925f-0683765661c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:30.232481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:30.233512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:30.234556 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:30.234556 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:30.234556 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:30.234556 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:30.241554 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 221/463] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:30.241554 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365b6c2-873d-49c5-925f-0683765661c7 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:30.275533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbcb2dd-5b61-4dec-8acf-a1e13acb7702 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:30.309801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfafed-343f-4f63-a859-384d2a765502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:30.310555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfafed-343f-4f63-a859-384d2a765502 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:30.310836 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfafed-343f-4f63-a859-384d2a765502 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:30.311108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfafed-343f-4f63-a859-384d2a765502 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:30.311846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfafed-343f-4f63-a859-384d2a765502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:30.312146 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfafed-343f-4f63-a859-384d2a765502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:30.313501 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfafed-343f-4f63-a859-384d2a765502 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:30.335528 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 243/464] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:30.335528 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfafed-343f-4f63-a859-384d2a765502 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:30.347823 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365b6c2-873d-49c5-925f-0683765661c7 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:30.385347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046f006-a787-4c1c-8b31-49a03f8dfdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:30.386150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046f006-a787-4c1c-8b31-49a03f8dfdbb None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:30.386435 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046f006-a787-4c1c-8b31-49a03f8dfdbb None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:30.386717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046f006-a787-4c1c-8b31-49a03f8dfdbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:30.397640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046f006-a787-4c1c-8b31-49a03f8dfdbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:30.399080 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046f006-a787-4c1c-8b31-49a03f8dfdbb None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-d046f006-a787-4c1c-8b31-49a03f8dfdbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:30.416441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:30.417354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:30.418249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:30.418249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:30.418249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:30.418249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:30.418249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 222/465] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:30.419141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dbd5d-fa1c-4ab7-9d6f-1741ebc1a4b3 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:30.419141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dbd5d-fa1c-4ab7-9d6f-1741ebc1a4b3 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:30.419772 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dbd5d-fa1c-4ab7-9d6f-1741ebc1a4b3 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:30.420812 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 223/466] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:30.422651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6dbd5d-fa1c-4ab7-9d6f-1741ebc1a4b3 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:30.435134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfafed-343f-4f63-a859-384d2a765502 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:30.468982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edd2ed-5777-4090-a30f-eee99b345b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:30.469593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edd2ed-5777-4090-a30f-eee99b345b9e None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:30.469854 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edd2ed-5777-4090-a30f-eee99b345b9e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:30.470130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edd2ed-5777-4090-a30f-eee99b345b9e None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:30.470839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edd2ed-5777-4090-a30f-eee99b345b9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:30.471143 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edd2ed-5777-4090-a30f-eee99b345b9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:30.472561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edd2ed-5777-4090-a30f-eee99b345b9e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:30.488207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 244/467] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:30.488207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edd2ed-5777-4090-a30f-eee99b345b9e None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:30.526334 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6dbd5d-fa1c-4ab7-9d6f-1741ebc1a4b3 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:30.557724 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e63d6-aaf4-420b-bd62-793a20841c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:30.558389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e63d6-aaf4-420b-bd62-793a20841c6d None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:30.558669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e63d6-aaf4-420b-bd62-793a20841c6d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:30.558945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e63d6-aaf4-420b-bd62-793a20841c6d None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:30.559673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e63d6-aaf4-420b-bd62-793a20841c6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:30.561079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e63d6-aaf4-420b-bd62-793a20841c6d None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:30.585432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 224/468] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:16:30 2022] POST /identity/v3/users => generated 248 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:16:30.587016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09493e96-143c-4da6-a59a-fbb1d6b642a5 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:30.587337 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09493e96-143c-4da6-a59a-fbb1d6b642a5 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:30.587619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09493e96-143c-4da6-a59a-fbb1d6b642a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:30.601381 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edd2ed-5777-4090-a30f-eee99b345b9e None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:30.605430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-09493e96-143c-4da6-a59a-fbb1d6b642a5 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:30.606197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-09493e96-143c-4da6-a59a-fbb1d6b642a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:30.651440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4c6c8-9acc-47ec-81b6-a15f3ec62fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:30.652498 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4c6c8-9acc-47ec-81b6-a15f3ec62fd1 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:30.653016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4c6c8-9acc-47ec-81b6-a15f3ec62fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:30.653496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4c6c8-9acc-47ec-81b6-a15f3ec62fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:30.654448 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4c6c8-9acc-47ec-81b6-a15f3ec62fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:30.655658 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4c6c8-9acc-47ec-81b6-a15f3ec62fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:30.657665 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4c6c8-9acc-47ec-81b6-a15f3ec62fd1 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:30.673251 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 245/469] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:30.675591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4c6c8-9acc-47ec-81b6-a15f3ec62fd1 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:30.693408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 225/470] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:30.694970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09493e96-143c-4da6-a59a-fbb1d6b642a5 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:30.783446 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4c6c8-9acc-47ec-81b6-a15f3ec62fd1 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:30.804483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09493e96-143c-4da6-a59a-fbb1d6b642a5 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:30.824855 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d0100-8b68-4ddb-bd21-473b8568697d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:30.825532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d0100-8b68-4ddb-bd21-473b8568697d None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:30.825816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d0100-8b68-4ddb-bd21-473b8568697d None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:30.826094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d0100-8b68-4ddb-bd21-473b8568697d None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:30.826786 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d0100-8b68-4ddb-bd21-473b8568697d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:30.828287 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d0100-8b68-4ddb-bd21-473b8568697d None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:30.847097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504c8ef-84aa-4d86-929b-08cb0f1b0561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:30.847846 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504c8ef-84aa-4d86-929b-08cb0f1b0561 None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:30.848132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504c8ef-84aa-4d86-929b-08cb0f1b0561 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:30.848413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504c8ef-84aa-4d86-929b-08cb0f1b0561 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:30.849115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504c8ef-84aa-4d86-929b-08cb0f1b0561 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:30.853169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 246/471] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:16:30 2022] POST /identity/v3/users => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:16:30.853327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504c8ef-84aa-4d86-929b-08cb0f1b0561 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:30.853432 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d0100-8b68-4ddb-bd21-473b8568697d None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:30.881191 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 226/472] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:16:30 2022] POST /identity/v3/users => generated 248 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:16:30.883527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204399d-8052-47ce-b3aa-d354ac850243 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:30.883966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204399d-8052-47ce-b3aa-d354ac850243 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:30.884357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204399d-8052-47ce-b3aa-d354ac850243 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:30.886125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 227/473] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:30.905205 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02713c-4781-4bda-b70b-4dd50884786e None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:30.905516 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02713c-4781-4bda-b70b-4dd50884786e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:30.905767 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02713c-4781-4bda-b70b-4dd50884786e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:30.930986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-be02713c-4781-4bda-b70b-4dd50884786e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:30.933088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-be02713c-4781-4bda-b70b-4dd50884786e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:30.982768 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d0100-8b68-4ddb-bd21-473b8568697d None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:31.014022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 228/474] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:31.021200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b5ce0-b255-434c-bb80-9a5f220adf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:31.022067 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b5ce0-b255-434c-bb80-9a5f220adf85 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:31.022355 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b5ce0-b255-434c-bb80-9a5f220adf85 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:31.022633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b5ce0-b255-434c-bb80-9a5f220adf85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:31.036752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b5ce0-b255-434c-bb80-9a5f220adf85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:31.038196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b5ce0-b255-434c-bb80-9a5f220adf85 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:31.048176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02713c-4781-4bda-b70b-4dd50884786e None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-639b5ce0-b255-434c-bb80-9a5f220adf85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:31.078281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:31.080449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:31.085131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:31.085131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:31.085131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:31.085131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:31.085131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 247/475] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:31.085131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b5ce0-b255-434c-bb80-9a5f220adf85 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:31.172290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02713c-4781-4bda-b70b-4dd50884786e None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:31.192572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b5ce0-b255-434c-bb80-9a5f220adf85 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:31.208855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1a446-020b-4963-98bc-8e5f699a9ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:31.209538 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1a446-020b-4963-98bc-8e5f699a9ebc None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:31.209823 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1a446-020b-4963-98bc-8e5f699a9ebc None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:31.210102 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1a446-020b-4963-98bc-8e5f699a9ebc None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:31.210854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1a446-020b-4963-98bc-8e5f699a9ebc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:31.212960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1a446-020b-4963-98bc-8e5f699a9ebc None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:31.222471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.sql.core [None req-6fc1a446-020b-4963-98bc-8e5f699a9ebc None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 18:16:31.222471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 18:16:31.222471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [parameters: {'id': '33f486f2cfe441c0a1a2537b01376a12', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 18:16:31.222471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79825) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 18:16:31.226806 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdba1d-f117-477a-ad13-3de0d7f51264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:31.227503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdba1d-f117-477a-ad13-3de0d7f51264 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:31.227784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdba1d-f117-477a-ad13-3de0d7f51264 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:31.228057 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdba1d-f117-477a-ad13-3de0d7f51264 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:31.228771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bdba1d-f117-477a-ad13-3de0d7f51264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:31.229074 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bdba1d-f117-477a-ad13-3de0d7f51264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:31.230427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bdba1d-f117-477a-ad13-3de0d7f51264 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-6fc1a446-020b-4963-98bc-8e5f699a9ebc None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application result = self._query(query) May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application conn.query(q) May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application result.read() May 09 18:16:31.241051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return ref.to_dict() May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 18:16:31.241990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application next(self.gen) May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application yield resource May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application next(self.gen) May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self.session.rollback() May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self.force_reraise() May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise self.value May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application session.commit() May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 18:16:31.243442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._prepare_impl() May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self.session.flush() May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._flush(objects) May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application compat.raise_( May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application flush_context.execute() May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rec.execute(self) May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 18:16:31.244674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 18:16:31.245745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application result = self._query(query) May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application conn.query(q) May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application result.read() May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 18:16:31.246727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [parameters: {'id': '33f486f2cfe441c0a1a2537b01376a12', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:31.248249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 18:16:31.250400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:31.252798 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 229/476] 10.222.0.25 () {60 vars in 1131 bytes} [Mon May 9 18:16:31 2022] POST /identity/v3/roles => generated 144 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:31.252798 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1a446-020b-4963-98bc-8e5f699a9ebc None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:31.253249 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 248/477] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:31.254942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdba1d-f117-477a-ad13-3de0d7f51264 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:31.368842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdba1d-f117-477a-ad13-3de0d7f51264 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:31.388588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1a446-020b-4963-98bc-8e5f699a9ebc None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:31.430582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97888f4f-6557-440e-acce-546da123ca1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:31.432843 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97888f4f-6557-440e-acce-546da123ca1c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:31.433148 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97888f4f-6557-440e-acce-546da123ca1c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:31.433410 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97888f4f-6557-440e-acce-546da123ca1c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:31.447139 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2210e28-95dc-4b5f-b585-d0e8ede3c948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:31.447828 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2210e28-95dc-4b5f-b585-d0e8ede3c948 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:31.448109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2210e28-95dc-4b5f-b585-d0e8ede3c948 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:31.448381 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2210e28-95dc-4b5f-b585-d0e8ede3c948 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:31.449074 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2210e28-95dc-4b5f-b585-d0e8ede3c948 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:31.450497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2210e28-95dc-4b5f-b585-d0e8ede3c948 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:31.463480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97888f4f-6557-440e-acce-546da123ca1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-97888f4f-6557-440e-acce-546da123ca1c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:31.472778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:31.473831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:31.475393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:31.475393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:31.475393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:31.475393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:31.475393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:31.475393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:31.475393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:31.475393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:31.475393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:31.475393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:31.475393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 18:16:31.475393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:31.475393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 230/478] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:16:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:31.484647 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 249/479] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:16:31 2022] POST /identity/v3/users => generated 249 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:16:31.485082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97888f4f-6557-440e-acce-546da123ca1c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:31.619269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97888f4f-6557-440e-acce-546da123ca1c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:31.666302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec18ea61-0aff-425d-8476-21b28d12d467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:31.666985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18ea61-0aff-425d-8476-21b28d12d467 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:31.667342 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18ea61-0aff-425d-8476-21b28d12d467 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:31.667628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18ea61-0aff-425d-8476-21b28d12d467 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:31.668376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec18ea61-0aff-425d-8476-21b28d12d467 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:31.668680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec18ea61-0aff-425d-8476-21b28d12d467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:31.670055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec18ea61-0aff-425d-8476-21b28d12d467 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:31.678571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 231/480] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:33.511665 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514b54c-ccd1-42e8-8e84-ad59c0e7dac6 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:33.512666 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514b54c-ccd1-42e8-8e84-ad59c0e7dac6 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:33.513167 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514b54c-ccd1-42e8-8e84-ad59c0e7dac6 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:33.514935 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 250/481] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:33.520536 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dffd0-674c-4f49-969d-c939afc791c1 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:33.521318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dffd0-674c-4f49-969d-c939afc791c1 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:33.521787 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dffd0-674c-4f49-969d-c939afc791c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:33.548451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-7c2dffd0-674c-4f49-969d-c939afc791c1 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:33.550671 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7c2dffd0-674c-4f49-969d-c939afc791c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:33.631757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 232/482] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:33.654897 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514b54c-ccd1-42e8-8e84-ad59c0e7dac6 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:33.767124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c23e95-fcf7-4832-86eb-f9faf3bd7df6 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:33.767779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c23e95-fcf7-4832-86eb-f9faf3bd7df6 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:33.768774 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c23e95-fcf7-4832-86eb-f9faf3bd7df6 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:33.770582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 233/483] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:33.775393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50896bd6-a66f-4560-b3c2-48a263903350 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:33.775937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50896bd6-a66f-4560-b3c2-48a263903350 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:33.776902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50896bd6-a66f-4560-b3c2-48a263903350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:33.796639 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-50896bd6-a66f-4560-b3c2-48a263903350 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:33.798077 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514b54c-ccd1-42e8-8e84-ad59c0e7dac6 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:33.798358 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-50896bd6-a66f-4560-b3c2-48a263903350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:33.848517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4b944-569c-4429-8ddb-20c3e9d2734e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:33.849953 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4b944-569c-4429-8ddb-20c3e9d2734e None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:33.850799 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4b944-569c-4429-8ddb-20c3e9d2734e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:33.851304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4b944-569c-4429-8ddb-20c3e9d2734e None admin] PATH_INFO: `/v3/roles/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:33.864346 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4b944-569c-4429-8ddb-20c3e9d2734e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:33.871251 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 234/484] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:33.872745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c6dad-02e9-4a7c-af3c-4694c23cb215 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:33.873132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c6dad-02e9-4a7c-af3c-4694c23cb215 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:33.873510 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c6dad-02e9-4a7c-af3c-4694c23cb215 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-28f4b944-569c-4429-8ddb-20c3e9d2734e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:33.873978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:33.875007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:33.876004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:33.876004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:33.876004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:33.876004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:33.876004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:33.876004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:33.876004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:33.876004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:33.876004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:33.876004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:33.876004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:33.876004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:33.877954 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 235/485] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:33.878432 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 251/486] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:33.888164 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4b944-569c-4429-8ddb-20c3e9d2734e None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:33.888559 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30895f48-5de9-471e-8bbc-f078fc1c2a86 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:33.888855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30895f48-5de9-471e-8bbc-f078fc1c2a86 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:33.889135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30895f48-5de9-471e-8bbc-f078fc1c2a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:33.912614 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-30895f48-5de9-471e-8bbc-f078fc1c2a86 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:33.913786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-30895f48-5de9-471e-8bbc-f078fc1c2a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:33.985487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 236/487] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:33.986431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30895f48-5de9-471e-8bbc-f078fc1c2a86 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:34.006970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4b944-569c-4429-8ddb-20c3e9d2734e None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:34.042153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c175088-d0fd-4cfd-bb09-688bdeb3eaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:34.043170 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c175088-d0fd-4cfd-bb09-688bdeb3eaa2 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.043690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c175088-d0fd-4cfd-bb09-688bdeb3eaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.044175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c175088-d0fd-4cfd-bb09-688bdeb3eaa2 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.045105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c175088-d0fd-4cfd-bb09-688bdeb3eaa2 None admin] RBAC: Adding query filter params (name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:34.045597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c175088-d0fd-4cfd-bb09-688bdeb3eaa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:34.047150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c175088-d0fd-4cfd-bb09-688bdeb3eaa2 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:34.060241 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 252/488] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:34.061400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c175088-d0fd-4cfd-bb09-688bdeb3eaa2 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:34.103379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30895f48-5de9-471e-8bbc-f078fc1c2a86 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:34.137078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442c82f-2c59-43b2-a028-23aa2f867657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:34.138114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442c82f-2c59-43b2-a028-23aa2f867657 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.138542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442c82f-2c59-43b2-a028-23aa2f867657 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.138984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442c82f-2c59-43b2-a028-23aa2f867657 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.150302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442c82f-2c59-43b2-a028-23aa2f867657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-7442c82f-2c59-43b2-a028-23aa2f867657 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:34.160039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:34.160940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:34.161726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:34.161726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:34.161726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:34.161726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:34.161726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:34.161726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:34.161726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:34.161726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:34.161726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:34.161726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:34.161726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:34.161726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:34.163921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c175088-d0fd-4cfd-bb09-688bdeb3eaa2 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:34.164338 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 237/489] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:34.169843 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442c82f-2c59-43b2-a028-23aa2f867657 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:34.198409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe5b41-b11a-4934-97b1-0718fae986c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:34.199221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe5b41-b11a-4934-97b1-0718fae986c5 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.199655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe5b41-b11a-4934-97b1-0718fae986c5 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.200170 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe5b41-b11a-4934-97b1-0718fae986c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.210889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe5b41-b11a-4934-97b1-0718fae986c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-fefe5b41-b11a-4934-97b1-0718fae986c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:34.218886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:34.219883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:34.220678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:34.220678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:34.220678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:34.220678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:34.220678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:34.220678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:34.220678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:34.220678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:34.220678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:34.220678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:34.220678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:34.220678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:34.227244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 253/490] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:34.228255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe5b41-b11a-4934-97b1-0718fae986c5 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:34.275024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442c82f-2c59-43b2-a028-23aa2f867657 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:34.309012 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde34803-aaee-4e75-8276-39b5210f484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:34.309768 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde34803-aaee-4e75-8276-39b5210f484f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.310143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde34803-aaee-4e75-8276-39b5210f484f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.310490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde34803-aaee-4e75-8276-39b5210f484f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.321870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde34803-aaee-4e75-8276-39b5210f484f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:34.323405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde34803-aaee-4e75-8276-39b5210f484f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:34.333485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe5b41-b11a-4934-97b1-0718fae986c5 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-bde34803-aaee-4e75-8276-39b5210f484f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:34.341285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:34.342166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:34.343003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:34.343003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:34.343003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:34.343003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:34.347501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 238/491] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:34.348473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde34803-aaee-4e75-8276-39b5210f484f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:34.372849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8c331-2285-4d42-8167-df3a369e2dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:34.373893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8c331-2285-4d42-8167-df3a369e2dfa None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.374340 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8c331-2285-4d42-8167-df3a369e2dfa None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.374786 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8c331-2285-4d42-8167-df3a369e2dfa None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.375759 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8c331-2285-4d42-8167-df3a369e2dfa None admin] RBAC: Adding query filter params (name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:34.376244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8c331-2285-4d42-8167-df3a369e2dfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:34.378503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8c331-2285-4d42-8167-df3a369e2dfa None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:34.387524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 254/492] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:34.388675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8c331-2285-4d42-8167-df3a369e2dfa None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:34.452479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde34803-aaee-4e75-8276-39b5210f484f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:34.490508 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c8ce8-28af-4a6f-a255-fa533f3af052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:34.491413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8ce8-28af-4a6f-a255-fa533f3af052 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.491702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8ce8-28af-4a6f-a255-fa533f3af052 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.491971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8ce8-28af-4a6f-a255-fa533f3af052 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.492713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c8ce8-28af-4a6f-a255-fa533f3af052 None admin] RBAC: Adding query filter params (name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:34.493014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c8ce8-28af-4a6f-a255-fa533f3af052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:34.494377 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c8ce8-28af-4a6f-a255-fa533f3af052 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:34.501376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8c331-2285-4d42-8167-df3a369e2dfa None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:34.507349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 239/493] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:34.507349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c8ce8-28af-4a6f-a255-fa533f3af052 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:34.537943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15f16c-d638-4d4a-b3e5-189c2c21dec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:34.538604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15f16c-d638-4d4a-b3e5-189c2c21dec9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.538882 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15f16c-d638-4d4a-b3e5-189c2c21dec9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.539152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15f16c-d638-4d4a-b3e5-189c2c21dec9 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.539884 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15f16c-d638-4d4a-b3e5-189c2c21dec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:34.540183 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15f16c-d638-4d4a-b3e5-189c2c21dec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:34.541632 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15f16c-d638-4d4a-b3e5-189c2c21dec9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:34.555291 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 255/494] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:34.557009 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a17c83-cf45-4047-aca2-ed36a88f0af7 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.557292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a17c83-cf45-4047-aca2-ed36a88f0af7 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.557564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a17c83-cf45-4047-aca2-ed36a88f0af7 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.559544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 256/495] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:34.560182 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a17c83-cf45-4047-aca2-ed36a88f0af7 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:34.602074 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c8ce8-28af-4a6f-a255-fa533f3af052 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:34.634067 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90791aeb-dbbc-4f6e-b9dd-5a54527bd273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:34.634863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90791aeb-dbbc-4f6e-b9dd-5a54527bd273 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.635142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90791aeb-dbbc-4f6e-b9dd-5a54527bd273 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.635444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90791aeb-dbbc-4f6e-b9dd-5a54527bd273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.645408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90791aeb-dbbc-4f6e-b9dd-5a54527bd273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:34.646853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90791aeb-dbbc-4f6e-b9dd-5a54527bd273 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:34.653448 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a17c83-cf45-4047-aca2-ed36a88f0af7 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-90791aeb-dbbc-4f6e-b9dd-5a54527bd273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:34.663054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:34.663990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:34.664857 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:34.664857 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:34.664857 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:34.664857 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:34.665436 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 240/496] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:34.666266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90791aeb-dbbc-4f6e-b9dd-5a54527bd273 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:34.685335 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8631f2-0cac-45b2-9e0a-c77c3d87d143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:34.686119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8631f2-0cac-45b2-9e0a-c77c3d87d143 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.686396 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8631f2-0cac-45b2-9e0a-c77c3d87d143 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.686664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8631f2-0cac-45b2-9e0a-c77c3d87d143 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.696788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8631f2-0cac-45b2-9e0a-c77c3d87d143 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:34.698193 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8631f2-0cac-45b2-9e0a-c77c3d87d143 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-ca8631f2-0cac-45b2-9e0a-c77c3d87d143 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:34.714692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:34.715597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:34.716444 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:34.716444 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:34.716444 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:34.716444 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:34.716444 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 257/497] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:34.717020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021be39-cf76-4238-a73b-0c5f637c60cf None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.717294 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021be39-cf76-4238-a73b-0c5f637c60cf None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.717561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021be39-cf76-4238-a73b-0c5f637c60cf None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.718784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 258/498] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:34.719822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb64a8d-620f-490d-848e-63c4d0599e18 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.720099 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb64a8d-620f-490d-848e-63c4d0599e18 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.720366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb64a8d-620f-490d-848e-63c4d0599e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.736464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-acb64a8d-620f-490d-848e-63c4d0599e18 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:34.738167 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-acb64a8d-620f-490d-848e-63c4d0599e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:34.765399 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90791aeb-dbbc-4f6e-b9dd-5a54527bd273 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:34.796327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbced302-09b2-4f31-af05-0675b507dac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:34.797109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbced302-09b2-4f31-af05-0675b507dac4 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.797386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbced302-09b2-4f31-af05-0675b507dac4 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.797657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbced302-09b2-4f31-af05-0675b507dac4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.800252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 259/499] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:34.801054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb64a8d-620f-490d-848e-63c4d0599e18 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:34.805434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbced302-09b2-4f31-af05-0675b507dac4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:34.806863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbced302-09b2-4f31-af05-0675b507dac4 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-bbced302-09b2-4f31-af05-0675b507dac4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:34.822611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:34.823778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:34.824676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:34.824676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:34.824676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:34.824676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:34.824676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:34.824676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:34.824676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:34.824676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:34.824676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 18:16:34.824676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:34.824676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 241/500] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:34.824676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbced302-09b2-4f31-af05-0675b507dac4 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:34.901514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb64a8d-620f-490d-848e-63c4d0599e18 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:34.917901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbced302-09b2-4f31-af05-0675b507dac4 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:34.933318 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e47a4-f7ab-482d-88ec-eba7678ebb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:34.933965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e47a4-f7ab-482d-88ec-eba7678ebb0a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.934241 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e47a4-f7ab-482d-88ec-eba7678ebb0a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.934509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e47a4-f7ab-482d-88ec-eba7678ebb0a None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.935237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e47a4-f7ab-482d-88ec-eba7678ebb0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:34.935538 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e47a4-f7ab-482d-88ec-eba7678ebb0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:34.936888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e47a4-f7ab-482d-88ec-eba7678ebb0a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:34.947337 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 260/501] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:34.948475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68934791-656f-4a44-a49f-ea27c21e6999 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.948757 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68934791-656f-4a44-a49f-ea27c21e6999 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.949028 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68934791-656f-4a44-a49f-ea27c21e6999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.949133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5109d-9bde-4256-af18-dae9a5775856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:34.949680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5109d-9bde-4256-af18-dae9a5775856 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:34.949956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5109d-9bde-4256-af18-dae9a5775856 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:34.950225 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5109d-9bde-4256-af18-dae9a5775856 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:34.950943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5109d-9bde-4256-af18-dae9a5775856 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:34.951286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5109d-9bde-4256-af18-dae9a5775856 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:34.952617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5109d-9bde-4256-af18-dae9a5775856 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:34.964331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 242/502] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:34.965101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-68934791-656f-4a44-a49f-ea27c21e6999 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:34.965434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5109d-9bde-4256-af18-dae9a5775856 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:34.966376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-68934791-656f-4a44-a49f-ea27c21e6999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:35.029172 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 261/503] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:35.029986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68934791-656f-4a44-a49f-ea27c21e6999 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:35.061480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5109d-9bde-4256-af18-dae9a5775856 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:35.093282 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96a128-69ba-40c4-a162-14b039b34e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:35.094009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96a128-69ba-40c4-a162-14b039b34e2d None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:35.094290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96a128-69ba-40c4-a162-14b039b34e2d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:35.094566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96a128-69ba-40c4-a162-14b039b34e2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:35.104657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96a128-69ba-40c4-a162-14b039b34e2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-8f96a128-69ba-40c4-a162-14b039b34e2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:35.112341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:35.113245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:35.114036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:35.114036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:35.114036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:35.114036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:35.114036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:35.114036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:35.114036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:35.114036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:35.114036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:35.114036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:35.114036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:35.114036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:35.114036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 243/504] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:35.114703 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96a128-69ba-40c4-a162-14b039b34e2d None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:35.136119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68934791-656f-4a44-a49f-ea27c21e6999 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:35.170602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275db17-4da0-4f7b-8d74-af70752fee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:35.171290 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275db17-4da0-4f7b-8d74-af70752fee27 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:35.171573 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275db17-4da0-4f7b-8d74-af70752fee27 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:35.171839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275db17-4da0-4f7b-8d74-af70752fee27 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:35.172656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275db17-4da0-4f7b-8d74-af70752fee27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:35.172963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275db17-4da0-4f7b-8d74-af70752fee27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:35.174392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275db17-4da0-4f7b-8d74-af70752fee27 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:35.209116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 262/505] 10.222.0.25 () {56 vars in 1137 bytes} [Mon May 9 18:16:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:35.209993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275db17-4da0-4f7b-8d74-af70752fee27 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:35.210532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96a128-69ba-40c4-a162-14b039b34e2d None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:35.242364 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba8e49-ec9d-4ded-9f52-8522aeb83222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:35.243148 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba8e49-ec9d-4ded-9f52-8522aeb83222 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:35.243461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba8e49-ec9d-4ded-9f52-8522aeb83222 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:35.243734 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba8e49-ec9d-4ded-9f52-8522aeb83222 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:35.250977 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba8e49-ec9d-4ded-9f52-8522aeb83222 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:35.252461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba8e49-ec9d-4ded-9f52-8522aeb83222 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-70ba8e49-ec9d-4ded-9f52-8522aeb83222 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:35.266485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:35.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:35.268919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:35.268919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:35.268919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:35.268919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:35.268919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:35.268919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:35.268919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:35.268919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:35.268919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 18:16:35.268919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:35.268919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 244/506] 10.222.0.25 () {56 vars in 1093 bytes} [Mon May 9 18:16:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:35.269696 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba8e49-ec9d-4ded-9f52-8522aeb83222 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:35.307324 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275db17-4da0-4f7b-8d74-af70752fee27 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:35.338485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0798f31-7400-4061-9616-b0ba593f7ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:35.339299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0798f31-7400-4061-9616-b0ba593f7ed0 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:35.339576 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0798f31-7400-4061-9616-b0ba593f7ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:35.339844 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0798f31-7400-4061-9616-b0ba593f7ed0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:35.347122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0798f31-7400-4061-9616-b0ba593f7ed0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:35.348518 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0798f31-7400-4061-9616-b0ba593f7ed0 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:35.362196 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba8e49-ec9d-4ded-9f52-8522aeb83222 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-a0798f31-7400-4061-9616-b0ba593f7ed0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:35.364161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:35.365023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:35.365890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:35.365890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:35.365890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:35.365890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:35.365890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:35.365890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:35.365890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:35.365890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:35.365890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 18:16:35.365890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:35.365890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 263/507] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:35.367285 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0798f31-7400-4061-9616-b0ba593f7ed0 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:35.393996 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b4ef1-8747-4e6a-bf6a-8c5462d8d9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:35.394829 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b4ef1-8747-4e6a-bf6a-8c5462d8d9ca None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:35.395105 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b4ef1-8747-4e6a-bf6a-8c5462d8d9ca None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:35.395405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b4ef1-8747-4e6a-bf6a-8c5462d8d9ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:35.405403 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b4ef1-8747-4e6a-bf6a-8c5462d8d9ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:35.406842 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b4ef1-8747-4e6a-bf6a-8c5462d8d9ca None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-ad2b4ef1-8747-4e6a-bf6a-8c5462d8d9ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:35.423046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:35.424078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:35.425608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:35.425608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:35.425608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:35.425608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:35.427318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 245/508] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:35.428158 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b4ef1-8747-4e6a-bf6a-8c5462d8d9ca None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:35.460602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0798f31-7400-4061-9616-b0ba593f7ed0 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:35.493944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39abe34-b8d1-43e3-a709-183b3cdbbf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:35.494598 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39abe34-b8d1-43e3-a709-183b3cdbbf23 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:35.494878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39abe34-b8d1-43e3-a709-183b3cdbbf23 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:35.495148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39abe34-b8d1-43e3-a709-183b3cdbbf23 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:35.495889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39abe34-b8d1-43e3-a709-183b3cdbbf23 None admin] RBAC: Adding query filter params (name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:35.496185 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39abe34-b8d1-43e3-a709-183b3cdbbf23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:35.497527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39abe34-b8d1-43e3-a709-183b3cdbbf23 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:35.505145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 264/509] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:35.505916 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39abe34-b8d1-43e3-a709-183b3cdbbf23 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:35.522655 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b4ef1-8747-4e6a-bf6a-8c5462d8d9ca None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:35.557242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269ae0b-b740-477e-b131-5b6f783af778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:35.557891 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269ae0b-b740-477e-b131-5b6f783af778 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:35.558171 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269ae0b-b740-477e-b131-5b6f783af778 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:35.558446 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269ae0b-b740-477e-b131-5b6f783af778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:35.597386 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39abe34-b8d1-43e3-a709-183b3cdbbf23 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:35.621412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269ae0b-b740-477e-b131-5b6f783af778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:35.622839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269ae0b-b740-477e-b131-5b6f783af778 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:35.628470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aeee3-0488-421c-86d4-13fe4d8b4e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:35.629123 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aeee3-0488-421c-86d4-13fe4d8b4e2f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:35.629400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aeee3-0488-421c-86d4-13fe4d8b4e2f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:35.629669 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aeee3-0488-421c-86d4-13fe4d8b4e2f None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:35.630481 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aeee3-0488-421c-86d4-13fe4d8b4e2f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:35.630778 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aeee3-0488-421c-86d4-13fe4d8b4e2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:35.632243 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aeee3-0488-421c-86d4-13fe4d8b4e2f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:35.648049 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 265/510] 10.222.0.25 () {56 vars in 1133 bytes} [Mon May 9 18:16:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:35.648823 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aeee3-0488-421c-86d4-13fe4d8b4e2f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:35.713906 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 246/511] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:35.714731 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269ae0b-b740-477e-b131-5b6f783af778 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:35.740275 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aeee3-0488-421c-86d4-13fe4d8b4e2f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:35.774124 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379c631-94f4-4d98-a93c-7fc2775ee9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:35.774781 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379c631-94f4-4d98-a93c-7fc2775ee9e3 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:35.775062 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379c631-94f4-4d98-a93c-7fc2775ee9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:35.775367 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379c631-94f4-4d98-a93c-7fc2775ee9e3 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:35.776179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379c631-94f4-4d98-a93c-7fc2775ee9e3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:35.776483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379c631-94f4-4d98-a93c-7fc2775ee9e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:35.777903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379c631-94f4-4d98-a93c-7fc2775ee9e3 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:35.794486 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 266/512] 10.222.0.25 () {56 vars in 1137 bytes} [Mon May 9 18:16:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:35.795314 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379c631-94f4-4d98-a93c-7fc2775ee9e3 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:35.811046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269ae0b-b740-477e-b131-5b6f783af778 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:35.842725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741e09b-5b10-430b-bd0b-617cfed5edc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:35.843405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741e09b-5b10-430b-bd0b-617cfed5edc9 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:35.843711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741e09b-5b10-430b-bd0b-617cfed5edc9 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:35.843987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741e09b-5b10-430b-bd0b-617cfed5edc9 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:35.844713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741e09b-5b10-430b-bd0b-617cfed5edc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:35.845019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741e09b-5b10-430b-bd0b-617cfed5edc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:35.846924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741e09b-5b10-430b-bd0b-617cfed5edc9 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:35.857615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 247/513] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:35.858327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741e09b-5b10-430b-bd0b-617cfed5edc9 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:35.891134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379c631-94f4-4d98-a93c-7fc2775ee9e3 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:35.922797 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561e8e0-d193-4bb8-9492-1c0f3331ebc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:35.923601 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561e8e0-d193-4bb8-9492-1c0f3331ebc2 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:35.923881 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561e8e0-d193-4bb8-9492-1c0f3331ebc2 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:35.924189 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561e8e0-d193-4bb8-9492-1c0f3331ebc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:35.933595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561e8e0-d193-4bb8-9492-1c0f3331ebc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:35.935009 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561e8e0-d193-4bb8-9492-1c0f3331ebc2 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:35.949372 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741e09b-5b10-430b-bd0b-617cfed5edc9 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-6561e8e0-d193-4bb8-9492-1c0f3331ebc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:35.951621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:35.952621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:35.953664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:35.953664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:35.953664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:35.953664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:35.955977 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 267/514] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:35.956692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561e8e0-d193-4bb8-9492-1c0f3331ebc2 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:35.983196 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10455b5-91ce-4604-b0b8-9de9b919d9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:35.983861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10455b5-91ce-4604-b0b8-9de9b919d9a1 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:35.984140 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10455b5-91ce-4604-b0b8-9de9b919d9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:35.984414 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10455b5-91ce-4604-b0b8-9de9b919d9a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:36.048195 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10455b5-91ce-4604-b0b8-9de9b919d9a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:36.049950 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10455b5-91ce-4604-b0b8-9de9b919d9a1 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:36.052793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561e8e0-d193-4bb8-9492-1c0f3331ebc2 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:36.086089 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1376840-3cf0-447b-8382-3c1458cc8e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:36.086874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1376840-3cf0-447b-8382-3c1458cc8e06 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:36.087152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1376840-3cf0-447b-8382-3c1458cc8e06 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:36.087453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1376840-3cf0-447b-8382-3c1458cc8e06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:36.098889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1376840-3cf0-447b-8382-3c1458cc8e06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:36.100370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1376840-3cf0-447b-8382-3c1458cc8e06 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-d1376840-3cf0-447b-8382-3c1458cc8e06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:36.116680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:36.117684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:36.118562 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:36.118562 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:36.118562 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:36.118562 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:36.118562 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 268/515] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:36.118562 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1376840-3cf0-447b-8382-3c1458cc8e06 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:36.146994 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 248/516] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:36.147880 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10455b5-91ce-4604-b0b8-9de9b919d9a1 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:36.215013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1376840-3cf0-447b-8382-3c1458cc8e06 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:36.246459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf6706-8f39-4997-afce-a5eeca10cef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:36.247098 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf6706-8f39-4997-afce-a5eeca10cef9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:36.247413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf6706-8f39-4997-afce-a5eeca10cef9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:36.247691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf6706-8f39-4997-afce-a5eeca10cef9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:36.248757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10455b5-91ce-4604-b0b8-9de9b919d9a1 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:36.281618 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef34886-9a5b-4159-9630-1f004f141b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:36.282293 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef34886-9a5b-4159-9630-1f004f141b83 None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:36.282576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef34886-9a5b-4159-9630-1f004f141b83 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:36.282851 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef34886-9a5b-4159-9630-1f004f141b83 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:36.283605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef34886-9a5b-4159-9630-1f004f141b83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:36.285032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef34886-9a5b-4159-9630-1f004f141b83 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:36.303716 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 249/517] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:16:36 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:16:36.304507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef34886-9a5b-4159-9630-1f004f141b83 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:36.311413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf6706-8f39-4997-afce-a5eeca10cef9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:36.312888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf6706-8f39-4997-afce-a5eeca10cef9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:36.423535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 269/518] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:36.423535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf6706-8f39-4997-afce-a5eeca10cef9 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:36.423805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef34886-9a5b-4159-9630-1f004f141b83 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:36.454632 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3174df2-ecdb-4325-9041-c68a39bc9e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:36.455358 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3174df2-ecdb-4325-9041-c68a39bc9e76 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:36.456179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3174df2-ecdb-4325-9041-c68a39bc9e76 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:36.456454 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3174df2-ecdb-4325-9041-c68a39bc9e76 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:36.457177 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3174df2-ecdb-4325-9041-c68a39bc9e76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:36.457481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3174df2-ecdb-4325-9041-c68a39bc9e76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:36.458872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3174df2-ecdb-4325-9041-c68a39bc9e76 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:36.471251 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 250/519] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:36.472111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3174df2-ecdb-4325-9041-c68a39bc9e76 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:36.535619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf6706-8f39-4997-afce-a5eeca10cef9 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:36.569378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ab582-fd74-4dc5-ab09-66f16f4aae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:36.570046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ab582-fd74-4dc5-ab09-66f16f4aae0a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:36.570328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ab582-fd74-4dc5-ab09-66f16f4aae0a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:36.570604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ab582-fd74-4dc5-ab09-66f16f4aae0a None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:36.571356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ab582-fd74-4dc5-ab09-66f16f4aae0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:36.571664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ab582-fd74-4dc5-ab09-66f16f4aae0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:36.572633 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3174df2-ecdb-4325-9041-c68a39bc9e76 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:36.573020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ab582-fd74-4dc5-ab09-66f16f4aae0a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:36.584585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 270/520] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:36.585640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ab582-fd74-4dc5-ab09-66f16f4aae0a None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:36.605777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59beed09-fcea-4662-802c-6953c8e89fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:36.606840 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beed09-fcea-4662-802c-6953c8e89fec None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:36.607331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beed09-fcea-4662-802c-6953c8e89fec None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:36.607796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beed09-fcea-4662-802c-6953c8e89fec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:36.618060 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59beed09-fcea-4662-802c-6953c8e89fec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:36.619716 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59beed09-fcea-4662-802c-6953c8e89fec None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-59beed09-fcea-4662-802c-6953c8e89fec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:36.637478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:36.638751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:36.639597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:36.639597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:36.639597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:36.639597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:36.644869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 251/521] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:36.644869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59beed09-fcea-4662-802c-6953c8e89fec None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:36.685332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ab582-fd74-4dc5-ab09-66f16f4aae0a None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:36.717724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18693201-dbb4-4560-99ff-fa9cace39fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:36.718516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18693201-dbb4-4560-99ff-fa9cace39fea None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:36.718793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18693201-dbb4-4560-99ff-fa9cace39fea None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:36.719120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18693201-dbb4-4560-99ff-fa9cace39fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:36.729148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18693201-dbb4-4560-99ff-fa9cace39fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:36.730667 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18693201-dbb4-4560-99ff-fa9cace39fea None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:36.744227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59beed09-fcea-4662-802c-6953c8e89fec None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-18693201-dbb4-4560-99ff-fa9cace39fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:36.747873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:36.748690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:36.749523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:36.749523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:36.749523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:36.749523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:36.751460 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 271/522] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:36.752114 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18693201-dbb4-4560-99ff-fa9cace39fea None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:36.778348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a11c4a-1bc5-456f-b12a-d2ade7aae8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:36.779142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a11c4a-1bc5-456f-b12a-d2ade7aae8e9 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:36.779453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a11c4a-1bc5-456f-b12a-d2ade7aae8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:36.779730 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a11c4a-1bc5-456f-b12a-d2ade7aae8e9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:36.786931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a11c4a-1bc5-456f-b12a-d2ade7aae8e9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:36.788401 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a11c4a-1bc5-456f-b12a-d2ade7aae8e9 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-13a11c4a-1bc5-456f-b12a-d2ade7aae8e9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:36.804180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:36.805089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:36.805960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:36.805960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:36.805960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:36.805960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:36.805960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:36.805960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:36.805960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:36.805960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:36.805960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 18:16:36.805960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:36.805960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 252/523] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:36.806514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a11c4a-1bc5-456f-b12a-d2ade7aae8e9 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:36.855508 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18693201-dbb4-4560-99ff-fa9cace39fea None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:36.889047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d7575-0c68-49f0-b014-b2b57f174ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:36.889838 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d7575-0c68-49f0-b014-b2b57f174ea4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:36.890121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d7575-0c68-49f0-b014-b2b57f174ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:36.890395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d7575-0c68-49f0-b014-b2b57f174ea4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:36.896938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d7575-0c68-49f0-b014-b2b57f174ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:36.898366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d7575-0c68-49f0-b014-b2b57f174ea4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:36.905816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a11c4a-1bc5-456f-b12a-d2ade7aae8e9 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-e55d7575-0c68-49f0-b014-b2b57f174ea4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:36.913033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:36.913970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:36.914821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:36.914821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:36.914821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:36.914821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:36.914821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:36.914821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:36.914821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:36.914821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:36.914821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 272/524] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:36.915682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d7575-0c68-49f0-b014-b2b57f174ea4 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:36.944558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c0f57-ec24-4bf9-a708-ba34334eab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:36.945243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c0f57-ec24-4bf9-a708-ba34334eab44 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:36.945527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c0f57-ec24-4bf9-a708-ba34334eab44 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:36.945802 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c0f57-ec24-4bf9-a708-ba34334eab44 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:36.946506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c0f57-ec24-4bf9-a708-ba34334eab44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:36.946811 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c0f57-ec24-4bf9-a708-ba34334eab44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:36.948219 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c0f57-ec24-4bf9-a708-ba34334eab44 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:36.959807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 253/525] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:36.961429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c0f57-ec24-4bf9-a708-ba34334eab44 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:37.018526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d7575-0c68-49f0-b014-b2b57f174ea4 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:37.051695 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3fda9-ac32-40ab-b875-bba6d83212b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:37.052351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3fda9-ac32-40ab-b875-bba6d83212b9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:37.052628 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3fda9-ac32-40ab-b875-bba6d83212b9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:37.052898 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3fda9-ac32-40ab-b875-bba6d83212b9 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:37.053594 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3fda9-ac32-40ab-b875-bba6d83212b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:37.053889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3fda9-ac32-40ab-b875-bba6d83212b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:37.055244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3fda9-ac32-40ab-b875-bba6d83212b9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:37.061110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c0f57-ec24-4bf9-a708-ba34334eab44 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:37.069345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 273/526] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:37.070162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3fda9-ac32-40ab-b875-bba6d83212b9 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:37.097076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7adf06-9386-4930-ad2e-42e63d03c2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:37.098128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7adf06-9386-4930-ad2e-42e63d03c2ec None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:37.098853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7adf06-9386-4930-ad2e-42e63d03c2ec None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:37.099581 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7adf06-9386-4930-ad2e-42e63d03c2ec None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:37.100698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7adf06-9386-4930-ad2e-42e63d03c2ec None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:37.101443 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7adf06-9386-4930-ad2e-42e63d03c2ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:37.103295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7adf06-9386-4930-ad2e-42e63d03c2ec None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:37.153718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 254/527] 10.222.0.25 () {56 vars in 1139 bytes} [Mon May 9 18:16:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:37.153718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7adf06-9386-4930-ad2e-42e63d03c2ec None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:37.171090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3fda9-ac32-40ab-b875-bba6d83212b9 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:37.203081 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49085f8-0698-4afa-b93b-a98d03da84a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:37.203689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49085f8-0698-4afa-b93b-a98d03da84a9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:37.203967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49085f8-0698-4afa-b93b-a98d03da84a9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:37.204240 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49085f8-0698-4afa-b93b-a98d03da84a9 None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:37.205002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49085f8-0698-4afa-b93b-a98d03da84a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:37.205347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49085f8-0698-4afa-b93b-a98d03da84a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:37.206746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49085f8-0698-4afa-b93b-a98d03da84a9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:37.223943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 274/528] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:37.224718 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49085f8-0698-4afa-b93b-a98d03da84a9 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:37.253596 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7adf06-9386-4930-ad2e-42e63d03c2ec None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:37.286262 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f26ef5-c0cc-43d4-af39-19ef4dc7e946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:37.287063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f26ef5-c0cc-43d4-af39-19ef4dc7e946 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:37.287377 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f26ef5-c0cc-43d4-af39-19ef4dc7e946 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:37.287662 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f26ef5-c0cc-43d4-af39-19ef4dc7e946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:37.294286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f26ef5-c0cc-43d4-af39-19ef4dc7e946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:37.295776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f26ef5-c0cc-43d4-af39-19ef4dc7e946 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-84f26ef5-c0cc-43d4-af39-19ef4dc7e946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:37.310307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:37.311135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:37.312027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:37.312027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:37.312027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:37.312027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:37.312027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:37.312027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:37.312027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:37.312027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:37.312027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 255/529] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:37.312491 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f26ef5-c0cc-43d4-af39-19ef4dc7e946 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:37.324126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49085f8-0698-4afa-b93b-a98d03da84a9 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:37.357966 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1767ca-6fa6-4559-8960-2dac717b31fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:37.358769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1767ca-6fa6-4559-8960-2dac717b31fa None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:37.359048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1767ca-6fa6-4559-8960-2dac717b31fa None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:37.359355 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1767ca-6fa6-4559-8960-2dac717b31fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:37.365745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1767ca-6fa6-4559-8960-2dac717b31fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:37.367246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1767ca-6fa6-4559-8960-2dac717b31fa None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-5f1767ca-6fa6-4559-8960-2dac717b31fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:37.380956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:37.381829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:37.383007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:37.383007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:37.383007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:37.383007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:37.383007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:37.383007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:37.383007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:37.383007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:37.387699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 275/530] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:37.388196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1767ca-6fa6-4559-8960-2dac717b31fa None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:37.418944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f26ef5-c0cc-43d4-af39-19ef4dc7e946 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:37.453793 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5f34c-8d43-4eed-8804-48d2fcc82e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:37.454374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5f34c-8d43-4eed-8804-48d2fcc82e46 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:37.454657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5f34c-8d43-4eed-8804-48d2fcc82e46 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:37.454930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5f34c-8d43-4eed-8804-48d2fcc82e46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:37.491526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1767ca-6fa6-4559-8960-2dac717b31fa None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:37.520466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5f34c-8d43-4eed-8804-48d2fcc82e46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:37.521916 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5f34c-8d43-4eed-8804-48d2fcc82e46 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:37.523977 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f9508-eff4-4fbc-8d2c-ae4ce95c3f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:37.524638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f9508-eff4-4fbc-8d2c-ae4ce95c3f91 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:37.524919 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f9508-eff4-4fbc-8d2c-ae4ce95c3f91 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:37.525199 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f9508-eff4-4fbc-8d2c-ae4ce95c3f91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:37.526020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f9508-eff4-4fbc-8d2c-ae4ce95c3f91 None admin] RBAC: Adding query filter params (role.id=7b2e521b45dc4a98adad996ca0027de6, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=77cf284e81ba4042b66c0bc189f1ae2b) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:37.526329 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f9508-eff4-4fbc-8d2c-ae4ce95c3f91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:37.527846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f9508-eff4-4fbc-8d2c-ae4ce95c3f91 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:37.533485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 276/531] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:37 2022] GET /identity/v3/role_assignments?user.id=77cf284e81ba4042b66c0bc189f1ae2b&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=7b2e521b45dc4a98adad996ca0027de6 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:37.534787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f9508-eff4-4fbc-8d2c-ae4ce95c3f91 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:37.630878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 256/532] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:37.632111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5f34c-8d43-4eed-8804-48d2fcc82e46 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:37.685758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f9508-eff4-4fbc-8d2c-ae4ce95c3f91 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:37.730860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337c7b8-0635-4914-82ac-a4a0730ba49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:37.731469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337c7b8-0635-4914-82ac-a4a0730ba49c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:37.731836 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337c7b8-0635-4914-82ac-a4a0730ba49c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:37.732115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337c7b8-0635-4914-82ac-a4a0730ba49c None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:37.732891 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337c7b8-0635-4914-82ac-a4a0730ba49c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:37.733201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337c7b8-0635-4914-82ac-a4a0730ba49c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:37.735387 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337c7b8-0635-4914-82ac-a4a0730ba49c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:37.747850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5f34c-8d43-4eed-8804-48d2fcc82e46 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:37.751243 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 277/533] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:37.754457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337c7b8-0635-4914-82ac-a4a0730ba49c None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:37.786187 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f471b8e-3133-43ee-bc84-7c937fa78fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:37.787240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f471b8e-3133-43ee-bc84-7c937fa78fb4 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:37.787748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f471b8e-3133-43ee-bc84-7c937fa78fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:37.788223 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f471b8e-3133-43ee-bc84-7c937fa78fb4 None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:37.789237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f471b8e-3133-43ee-bc84-7c937fa78fb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:37.789865 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f471b8e-3133-43ee-bc84-7c937fa78fb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:37.791570 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f471b8e-3133-43ee-bc84-7c937fa78fb4 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:37.810075 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 257/534] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:37.811194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f471b8e-3133-43ee-bc84-7c937fa78fb4 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:37.881626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337c7b8-0635-4914-82ac-a4a0730ba49c None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:37.912590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f471b8e-3133-43ee-bc84-7c937fa78fb4 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:37.921724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ea131-d1fe-429d-bd4e-798168f4de2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:37.922832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ea131-d1fe-429d-bd4e-798168f4de2e None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:37.923343 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ea131-d1fe-429d-bd4e-798168f4de2e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:37.923818 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ea131-d1fe-429d-bd4e-798168f4de2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:37.934379 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ea131-d1fe-429d-bd4e-798168f4de2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:37.936193 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ea131-d1fe-429d-bd4e-798168f4de2e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:37.950160 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5834fc5-9068-449d-874a-a86595603d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:37.950812 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5834fc5-9068-449d-874a-a86595603d2b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:37.951089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5834fc5-9068-449d-874a-a86595603d2b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:37.951404 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5834fc5-9068-449d-874a-a86595603d2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:37.952222 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5834fc5-9068-449d-874a-a86595603d2b None admin] RBAC: Adding query filter params (role.id=7b2e521b45dc4a98adad996ca0027de6, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=f769ee9f31354e5296645fb1e3bd84b6) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:37.952528 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5834fc5-9068-449d-874a-a86595603d2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:37.953933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5834fc5-9068-449d-874a-a86595603d2b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-072ea131-d1fe-429d-bd4e-798168f4de2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:37.956969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:37.957817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:37.958686 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:37.958686 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:37.958686 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:37.958686 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:37.961092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 278/535] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:37.962673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ea131-d1fe-429d-bd4e-798168f4de2e None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:37.967278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 258/536] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:37 2022] GET /identity/v3/role_assignments?user.id=f769ee9f31354e5296645fb1e3bd84b6&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=7b2e521b45dc4a98adad996ca0027de6 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:37.969038 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5834fc5-9068-449d-874a-a86595603d2b None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:38.170977 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5834fc5-9068-449d-874a-a86595603d2b None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:38.173119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ea131-d1fe-429d-bd4e-798168f4de2e None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:38.206634 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2558a60-5bb3-4c5e-b6cc-867310c102fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:38.207744 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2558a60-5bb3-4c5e-b6cc-867310c102fb None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:38.208231 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2558a60-5bb3-4c5e-b6cc-867310c102fb None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:38.208707 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2558a60-5bb3-4c5e-b6cc-867310c102fb None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:38.209684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2558a60-5bb3-4c5e-b6cc-867310c102fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:38.210255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2558a60-5bb3-4c5e-b6cc-867310c102fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:38.211866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2558a60-5bb3-4c5e-b6cc-867310c102fb None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:38.224368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 259/537] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:38.231250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2558a60-5bb3-4c5e-b6cc-867310c102fb None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:38.248612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d230f-59c8-4ac5-8f73-33d9db7b9905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:38.249675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d230f-59c8-4ac5-8f73-33d9db7b9905 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:38.253011 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d230f-59c8-4ac5-8f73-33d9db7b9905 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:38.253521 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d230f-59c8-4ac5-8f73-33d9db7b9905 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:38.254612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d230f-59c8-4ac5-8f73-33d9db7b9905 None admin] RBAC: Adding query filter params (role.id=7b2e521b45dc4a98adad996ca0027de6, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=4bda352a10f241ca96d5308a55238d57) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:38.256640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d230f-59c8-4ac5-8f73-33d9db7b9905 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:38.259149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d230f-59c8-4ac5-8f73-33d9db7b9905 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:38.266133 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 279/538] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:37 2022] GET /identity/v3/role_assignments?user.id=4bda352a10f241ca96d5308a55238d57&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=7b2e521b45dc4a98adad996ca0027de6 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:38.358521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2558a60-5bb3-4c5e-b6cc-867310c102fb None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:38.396391 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01f6df-c180-4985-989b-4d45df72299e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:38.397575 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01f6df-c180-4985-989b-4d45df72299e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:38.398077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01f6df-c180-4985-989b-4d45df72299e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:38.398550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01f6df-c180-4985-989b-4d45df72299e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:38.405686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01f6df-c180-4985-989b-4d45df72299e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:38.407368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01f6df-c180-4985-989b-4d45df72299e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-ea01f6df-c180-4985-989b-4d45df72299e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:38.422235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:38.423123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:38.424619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:38.424619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:38.424619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:38.424619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:38.424619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:38.424619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:38.424619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:38.424619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:38.427226 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 260/539] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:38.436298 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d230f-59c8-4ac5-8f73-33d9db7b9905 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:38.559505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d230f-59c8-4ac5-8f73-33d9db7b9905 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:38.594177 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da931367-ca90-43f3-9d1c-7c47eb494193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:38.595271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da931367-ca90-43f3-9d1c-7c47eb494193 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:38.595783 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da931367-ca90-43f3-9d1c-7c47eb494193 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:38.596865 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da931367-ca90-43f3-9d1c-7c47eb494193 None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:38.597912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da931367-ca90-43f3-9d1c-7c47eb494193 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:38.598448 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da931367-ca90-43f3-9d1c-7c47eb494193 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:38.600164 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da931367-ca90-43f3-9d1c-7c47eb494193 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:38.614150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 280/540] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:38.619797 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01f6df-c180-4985-989b-4d45df72299e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:38.720160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b312d08-5528-45aa-a909-fc2cbd97476f None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:38.721152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b312d08-5528-45aa-a909-fc2cbd97476f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:38.721654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b312d08-5528-45aa-a909-fc2cbd97476f None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:38.724397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 281/541] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:38.730225 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550a755-776b-4107-89e0-95bf9e4d1341 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:38.730739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550a755-776b-4107-89e0-95bf9e4d1341 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:38.731301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550a755-776b-4107-89e0-95bf9e4d1341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:38.734772 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01f6df-c180-4985-989b-4d45df72299e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:38.750538 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9550a755-776b-4107-89e0-95bf9e4d1341 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:38.753033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9550a755-776b-4107-89e0-95bf9e4d1341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:38.788424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f92237-caa7-434e-abd2-96fd28d88fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:38.789091 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f92237-caa7-434e-abd2-96fd28d88fdc None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:38.789368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f92237-caa7-434e-abd2-96fd28d88fdc None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:38.789644 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f92237-caa7-434e-abd2-96fd28d88fdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:38.790479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f92237-caa7-434e-abd2-96fd28d88fdc None admin] RBAC: Adding query filter params (role.id=7b2e521b45dc4a98adad996ca0027de6, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=6b2b6318776b422d9701351ba6b4ee27) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:38.790793 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f92237-caa7-434e-abd2-96fd28d88fdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:38.792250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f92237-caa7-434e-abd2-96fd28d88fdc None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:38.809554 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 261/542] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:38 2022] GET /identity/v3/role_assignments?user.id=6b2b6318776b422d9701351ba6b4ee27&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=7b2e521b45dc4a98adad996ca0027de6 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:38.828079 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 282/543] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:38.853260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f92237-caa7-434e-abd2-96fd28d88fdc None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:39.003972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f92237-caa7-434e-abd2-96fd28d88fdc None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:39.053661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ce58c-fa60-44f3-877b-d3d2aa7d5f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:39.054471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ce58c-fa60-44f3-877b-d3d2aa7d5f47 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:39.054751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ce58c-fa60-44f3-877b-d3d2aa7d5f47 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:39.055033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ce58c-fa60-44f3-877b-d3d2aa7d5f47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:39.067047 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ce58c-fa60-44f3-877b-d3d2aa7d5f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-a04ce58c-fa60-44f3-877b-d3d2aa7d5f47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:39.074766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:39.075678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:39.076461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:39.076461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:39.076461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:39.076461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:39.076461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:39.076461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:39.076461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:39.076461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:39.076461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:39.076461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:39.076461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:39.076461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:39.076461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 262/544] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:39.084090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550a755-776b-4107-89e0-95bf9e4d1341 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:39.212990 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550a755-776b-4107-89e0-95bf9e4d1341 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:39.262992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd28d6a-bcdf-4090-a378-b29f01e27e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:39.263613 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd28d6a-bcdf-4090-a378-b29f01e27e08 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:39.263895 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd28d6a-bcdf-4090-a378-b29f01e27e08 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:39.264169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd28d6a-bcdf-4090-a378-b29f01e27e08 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:39.264914 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd28d6a-bcdf-4090-a378-b29f01e27e08 None admin] RBAC: Adding query filter params (name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:39.265218 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd28d6a-bcdf-4090-a378-b29f01e27e08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:39.266708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd28d6a-bcdf-4090-a378-b29f01e27e08 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:39.278530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 283/545] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:39.284421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ce58c-fa60-44f3-877b-d3d2aa7d5f47 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:39.429879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ce58c-fa60-44f3-877b-d3d2aa7d5f47 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:39.476766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499405e-f10d-4480-91ed-7049bdbf6519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:39.477567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499405e-f10d-4480-91ed-7049bdbf6519 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:39.477851 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499405e-f10d-4480-91ed-7049bdbf6519 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:39.478128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499405e-f10d-4480-91ed-7049bdbf6519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:39.491218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499405e-f10d-4480-91ed-7049bdbf6519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:39.492656 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499405e-f10d-4480-91ed-7049bdbf6519 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-7499405e-f10d-4480-91ed-7049bdbf6519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:39.513833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:39.514700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:39.515587 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:39.515587 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:39.515587 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:39.515587 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:39.517171 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 263/546] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:39.522400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd28d6a-bcdf-4090-a378-b29f01e27e08 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:39.637254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd28d6a-bcdf-4090-a378-b29f01e27e08 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:39.670547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aef84-fc0c-4cf3-9870-b2f6febb35d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:39.671565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aef84-fc0c-4cf3-9870-b2f6febb35d4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:39.672052 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aef84-fc0c-4cf3-9870-b2f6febb35d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:39.672530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aef84-fc0c-4cf3-9870-b2f6febb35d4 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:39.673440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65aef84-fc0c-4cf3-9870-b2f6febb35d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:39.674544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65aef84-fc0c-4cf3-9870-b2f6febb35d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:39.676132 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65aef84-fc0c-4cf3-9870-b2f6febb35d4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:39.688478 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 284/547] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:39.694760 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499405e-f10d-4480-91ed-7049bdbf6519 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:39.851828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499405e-f10d-4480-91ed-7049bdbf6519 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:39.897382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31c79c-4ce0-424e-b509-e435b2275db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:39.898190 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31c79c-4ce0-424e-b509-e435b2275db2 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:39.898477 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31c79c-4ce0-424e-b509-e435b2275db2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:39.898755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31c79c-4ce0-424e-b509-e435b2275db2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:39.911385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31c79c-4ce0-424e-b509-e435b2275db2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:39.912860 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31c79c-4ce0-424e-b509-e435b2275db2 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-1c31c79c-4ce0-424e-b509-e435b2275db2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:39.933582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:39.934539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:39.935463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:39.935463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:39.935463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:39.935463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:39.935463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:39.935463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:39.935463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:39.935463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:39.935463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 18:16:39.935463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:39.937408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 264/548] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:16:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:39.942122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aef84-fc0c-4cf3-9870-b2f6febb35d4 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:39.992054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d73f9-4442-4472-9f8c-aee1e705477a None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:39.992359 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d73f9-4442-4472-9f8c-aee1e705477a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:39.992608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d73f9-4442-4472-9f8c-aee1e705477a None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:39.999268 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 265/549] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:40.008550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96fcb-73e9-486a-bf8b-eff4f01df916 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:40.008839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96fcb-73e9-486a-bf8b-eff4f01df916 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:40.009118 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96fcb-73e9-486a-bf8b-eff4f01df916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:40.048179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-d8d96fcb-73e9-486a-bf8b-eff4f01df916 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:40.049422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-d8d96fcb-73e9-486a-bf8b-eff4f01df916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:40.074995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aef84-fc0c-4cf3-9870-b2f6febb35d4 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:40.109020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9232824-e530-4c48-a52f-27d98499d392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:40.109707 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9232824-e530-4c48-a52f-27d98499d392 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:40.110078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9232824-e530-4c48-a52f-27d98499d392 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:40.110449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9232824-e530-4c48-a52f-27d98499d392 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:40.111304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9232824-e530-4c48-a52f-27d98499d392 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:40.111613 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9232824-e530-4c48-a52f-27d98499d392 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:40.113026 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9232824-e530-4c48-a52f-27d98499d392 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:40.131731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 285/550] 10.222.0.25 () {56 vars in 1135 bytes} [Mon May 9 18:16:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:40.138031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9232824-e530-4c48-a52f-27d98499d392 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:40.176687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 266/551] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:40.230353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d96fcb-73e9-486a-bf8b-eff4f01df916 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:40.300155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9232824-e530-4c48-a52f-27d98499d392 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:40.338697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88d0cf-1948-40bf-aab9-2ea7e53a9c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:40.339579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d0cf-1948-40bf-aab9-2ea7e53a9c6e None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:40.339724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d0cf-1948-40bf-aab9-2ea7e53a9c6e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:40.339931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d0cf-1948-40bf-aab9-2ea7e53a9c6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:40.374830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d96fcb-73e9-486a-bf8b-eff4f01df916 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:40.415086 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9cec4-ac40-47a3-8abe-dd6c5998ab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:40.416316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9cec4-ac40-47a3-8abe-dd6c5998ab83 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:40.416790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9cec4-ac40-47a3-8abe-dd6c5998ab83 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:40.417249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9cec4-ac40-47a3-8abe-dd6c5998ab83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:40.434340 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88d0cf-1948-40bf-aab9-2ea7e53a9c6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:40.436276 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88d0cf-1948-40bf-aab9-2ea7e53a9c6e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:40.460299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f9cec4-ac40-47a3-8abe-dd6c5998ab83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-03f9cec4-ac40-47a3-8abe-dd6c5998ab83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:40.478181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:40.479136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:40.484295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:40.484295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:40.484295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:40.484295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:40.484295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:40.484295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:40.484295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:40.484295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:40.484295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:40.484295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:40.484295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:40.484295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:40.484295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 267/552] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:40.490889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9cec4-ac40-47a3-8abe-dd6c5998ab83 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:40.559008 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 286/553] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:40.567739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88d0cf-1948-40bf-aab9-2ea7e53a9c6e None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:40.664843 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9cec4-ac40-47a3-8abe-dd6c5998ab83 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:40.683099 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88d0cf-1948-40bf-aab9-2ea7e53a9c6e None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:40.702248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be93bc-474f-46b3-b7b2-2246b49b6d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:40.703285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be93bc-474f-46b3-b7b2-2246b49b6d18 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:40.703742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be93bc-474f-46b3-b7b2-2246b49b6d18 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:40.704201 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be93bc-474f-46b3-b7b2-2246b49b6d18 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:40.705116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be93bc-474f-46b3-b7b2-2246b49b6d18 None admin] RBAC: Adding query filter params (name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:40.705619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be93bc-474f-46b3-b7b2-2246b49b6d18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:40.707425 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be93bc-474f-46b3-b7b2-2246b49b6d18 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:40.717171 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 268/554] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:40.723411 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fac539-ae0e-4a7f-8f39-296dac068cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:40.724213 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fac539-ae0e-4a7f-8f39-296dac068cad None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:40.724339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47489d9-9c9f-457b-876f-76dee9a7918d None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:40.724490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fac539-ae0e-4a7f-8f39-296dac068cad None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:40.724766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fac539-ae0e-4a7f-8f39-296dac068cad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:40.725111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47489d9-9c9f-457b-876f-76dee9a7918d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:40.725548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47489d9-9c9f-457b-876f-76dee9a7918d None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:40.727446 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 269/555] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:40.728357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47489d9-9c9f-457b-876f-76dee9a7918d None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:40.737424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fac539-ae0e-4a7f-8f39-296dac068cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:40.738881 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fac539-ae0e-4a7f-8f39-296dac068cad None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-80fac539-ae0e-4a7f-8f39-296dac068cad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:40.762830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:40.763771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:40.764732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:40.764732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:40.764732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:40.764732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:40.764732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 287/556] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:40.765753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6114-7127-4d0a-a314-3e88e51f9275 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:40.766046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6114-7127-4d0a-a314-3e88e51f9275 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:40.766331 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6114-7127-4d0a-a314-3e88e51f9275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:40.788506 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-e07f6114-7127-4d0a-a314-3e88e51f9275 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:40.790171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e07f6114-7127-4d0a-a314-3e88e51f9275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:40.856222 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47489d9-9c9f-457b-876f-76dee9a7918d None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:40.859224 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 288/557] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:40.860358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f6114-7127-4d0a-a314-3e88e51f9275 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:40.900547 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaddd5d-3d1f-4922-9176-0c5ace76f2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:40.902202 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaddd5d-3d1f-4922-9176-0c5ace76f2ea None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:40.902493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaddd5d-3d1f-4922-9176-0c5ace76f2ea None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:40.902772 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaddd5d-3d1f-4922-9176-0c5ace76f2ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:40.917390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaddd5d-3d1f-4922-9176-0c5ace76f2ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:40.918760 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaddd5d-3d1f-4922-9176-0c5ace76f2ea None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-5aaddd5d-3d1f-4922-9176-0c5ace76f2ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:40.947330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:40.948272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:40.949770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:40.949770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:40.949770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:40.949770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:40.949770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 270/558] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:40.954435 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaddd5d-3d1f-4922-9176-0c5ace76f2ea None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:40.982501 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f6114-7127-4d0a-a314-3e88e51f9275 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:41.024477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab45e07-c3b9-44ac-9305-40d9efc87a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:41.025311 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab45e07-c3b9-44ac-9305-40d9efc87a23 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:41.025748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab45e07-c3b9-44ac-9305-40d9efc87a23 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:41.026335 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab45e07-c3b9-44ac-9305-40d9efc87a23 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:41.026845 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab45e07-c3b9-44ac-9305-40d9efc87a23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:41.027228 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab45e07-c3b9-44ac-9305-40d9efc87a23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:41.028657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab45e07-c3b9-44ac-9305-40d9efc87a23 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:41.044648 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 289/559] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:41.045399 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab45e07-c3b9-44ac-9305-40d9efc87a23 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:41.087126 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaddd5d-3d1f-4922-9176-0c5ace76f2ea None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:41.129195 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb8a30-89d0-4fc2-89e3-53e666ebc460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:41.130598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb8a30-89d0-4fc2-89e3-53e666ebc460 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:41.131083 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb8a30-89d0-4fc2-89e3-53e666ebc460 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:41.131579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb8a30-89d0-4fc2-89e3-53e666ebc460 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:41.145412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb8a30-89d0-4fc2-89e3-53e666ebc460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-10eb8a30-89d0-4fc2-89e3-53e666ebc460 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:41.156394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:41.157473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:41.158260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:41.158260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:41.158260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:41.158260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:41.158260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:41.158260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:41.158260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:41.158260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:41.158260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:41.158260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:41.158260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:41.158260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:41.165432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 271/560] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:41.165432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb8a30-89d0-4fc2-89e3-53e666ebc460 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:41.180747 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab45e07-c3b9-44ac-9305-40d9efc87a23 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:41.224741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac716-c146-4c61-a00d-511321c7be65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:41.226326 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac716-c146-4c61-a00d-511321c7be65 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:41.226802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac716-c146-4c61-a00d-511321c7be65 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:41.227331 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac716-c146-4c61-a00d-511321c7be65 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:41.228775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccac716-c146-4c61-a00d-511321c7be65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:41.229573 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccac716-c146-4c61-a00d-511321c7be65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:41.231191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccac716-c146-4c61-a00d-511321c7be65 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:41.248867 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 290/561] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:41.250162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac716-c146-4c61-a00d-511321c7be65 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:41.280140 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb8a30-89d0-4fc2-89e3-53e666ebc460 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:41.329523 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0c089-a340-4053-93bf-a55fa6479b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:41.332222 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0c089-a340-4053-93bf-a55fa6479b8a None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:41.332720 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0c089-a340-4053-93bf-a55fa6479b8a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:41.333354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0c089-a340-4053-93bf-a55fa6479b8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:41.342464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0c089-a340-4053-93bf-a55fa6479b8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:41.344799 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0c089-a340-4053-93bf-a55fa6479b8a None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:41.363585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac716-c146-4c61-a00d-511321c7be65 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-cda0c089-a340-4053-93bf-a55fa6479b8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:41.364657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:41.365494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:41.366895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:41.366895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:41.366895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:41.366895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:41.366895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:41.366895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:41.366895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:41.366895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:41.374629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 272/562] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:41.374629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0c089-a340-4053-93bf-a55fa6479b8a None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:41.407656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6bee2-d5e5-4402-88d3-566fec5b768c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:41.408419 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6bee2-d5e5-4402-88d3-566fec5b768c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:41.409229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6bee2-d5e5-4402-88d3-566fec5b768c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:41.409502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6bee2-d5e5-4402-88d3-566fec5b768c None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:41.410441 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6bee2-d5e5-4402-88d3-566fec5b768c None admin] RBAC: Adding query filter params (name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:41.410570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6bee2-d5e5-4402-88d3-566fec5b768c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:41.412006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6bee2-d5e5-4402-88d3-566fec5b768c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:41.424982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 291/563] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:41.424982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6bee2-d5e5-4402-88d3-566fec5b768c None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:41.487639 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0c089-a340-4053-93bf-a55fa6479b8a None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:41.518723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6bee2-d5e5-4402-88d3-566fec5b768c None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:41.520522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314ae7-ea79-422a-9cb5-dcc3271692f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:41.521315 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314ae7-ea79-422a-9cb5-dcc3271692f5 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:41.521597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314ae7-ea79-422a-9cb5-dcc3271692f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:41.521869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314ae7-ea79-422a-9cb5-dcc3271692f5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:41.528966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c314ae7-ea79-422a-9cb5-dcc3271692f5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:41.530507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c314ae7-ea79-422a-9cb5-dcc3271692f5 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-5c314ae7-ea79-422a-9cb5-dcc3271692f5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:41.544815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:41.545753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:41.546947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:41.546947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:41.546947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:41.546947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:41.546947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:41.546947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:41.546947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:41.546947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:41.546947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 18:16:41.546947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:41.546947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 273/564] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:41.547790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b26ed4-bd1b-48d5-b578-696effad739f None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:41.547790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b26ed4-bd1b-48d5-b578-696effad739f None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:41.547927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b26ed4-bd1b-48d5-b578-696effad739f None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:41.549327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 274/565] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:41.551948 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b26ed4-bd1b-48d5-b578-696effad739f None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:41.552075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb5bee-5cdc-4847-82d7-8928578efc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:41.552631 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb5bee-5cdc-4847-82d7-8928578efc93 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:41.552970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb5bee-5cdc-4847-82d7-8928578efc93 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:41.553254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb5bee-5cdc-4847-82d7-8928578efc93 None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:41.554064 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb5bee-5cdc-4847-82d7-8928578efc93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:41.554374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb5bee-5cdc-4847-82d7-8928578efc93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:41.555813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb5bee-5cdc-4847-82d7-8928578efc93 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:41.568012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 292/566] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:41.569261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2b36d-b837-469c-85fc-b5dfa35f5a40 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:41.569540 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2b36d-b837-469c-85fc-b5dfa35f5a40 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:41.569808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2b36d-b837-469c-85fc-b5dfa35f5a40 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:41.570993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 293/567] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:41.571688 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2b36d-b837-469c-85fc-b5dfa35f5a40 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:41.654020 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b26ed4-bd1b-48d5-b578-696effad739f None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:41.664200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2b36d-b837-469c-85fc-b5dfa35f5a40 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:41.685318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea41f7d-afea-40f6-a53d-6da5cc2ed338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:41.686103 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea41f7d-afea-40f6-a53d-6da5cc2ed338 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:41.686385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea41f7d-afea-40f6-a53d-6da5cc2ed338 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:41.686660 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea41f7d-afea-40f6-a53d-6da5cc2ed338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:41.695765 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224d9f7-4fc4-4c19-b7b1-01e06cfdf5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:41.696417 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224d9f7-4fc4-4c19-b7b1-01e06cfdf5f7 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:41.696696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224d9f7-4fc4-4c19-b7b1-01e06cfdf5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:41.696849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea41f7d-afea-40f6-a53d-6da5cc2ed338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:41.696969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224d9f7-4fc4-4c19-b7b1-01e06cfdf5f7 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:41.697809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224d9f7-4fc4-4c19-b7b1-01e06cfdf5f7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:41.698113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224d9f7-4fc4-4c19-b7b1-01e06cfdf5f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:41.698267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea41f7d-afea-40f6-a53d-6da5cc2ed338 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:41.699517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224d9f7-4fc4-4c19-b7b1-01e06cfdf5f7 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-9ea41f7d-afea-40f6-a53d-6da5cc2ed338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:41.715501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:41.716445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:41.721411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:41.721411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:41.721411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:41.721411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:41.721640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 294/568] 10.222.0.25 () {56 vars in 1137 bytes} [Mon May 9 18:16:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:41.721753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 275/569] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:41.721822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e132ecb-dfc9-4ca1-8483-c349eeafd1cc None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:41.721822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e132ecb-dfc9-4ca1-8483-c349eeafd1cc None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:41.721943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea41f7d-afea-40f6-a53d-6da5cc2ed338 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:41.722019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e132ecb-dfc9-4ca1-8483-c349eeafd1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:41.736212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2e132ecb-dfc9-4ca1-8483-c349eeafd1cc None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:41.737399 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2e132ecb-dfc9-4ca1-8483-c349eeafd1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:41.799265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 295/570] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:41.800422 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829665f-c9ae-422f-8c63-b62c5640add1 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:41.800703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829665f-c9ae-422f-8c63-b62c5640add1 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:41.800973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829665f-c9ae-422f-8c63-b62c5640add1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:41.813029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea41f7d-afea-40f6-a53d-6da5cc2ed338 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:41.823961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9829665f-c9ae-422f-8c63-b62c5640add1 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:41.825096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9829665f-c9ae-422f-8c63-b62c5640add1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:41.845979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40aa674-d28a-4671-8a1f-2631e49492e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:41.846631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40aa674-d28a-4671-8a1f-2631e49492e1 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:41.846913 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40aa674-d28a-4671-8a1f-2631e49492e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:41.847211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40aa674-d28a-4671-8a1f-2631e49492e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:41.848029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40aa674-d28a-4671-8a1f-2631e49492e1 None admin] RBAC: Adding query filter params (role.id=7b2e521b45dc4a98adad996ca0027de6, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=ec2db3aa2b7d41f89314fec960d898b7) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:41.848340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40aa674-d28a-4671-8a1f-2631e49492e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:41.849737 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40aa674-d28a-4671-8a1f-2631e49492e1 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:41.855057 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 276/571] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:41 2022] GET /identity/v3/role_assignments?user.id=ec2db3aa2b7d41f89314fec960d898b7&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=7b2e521b45dc4a98adad996ca0027de6 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:41.855666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40aa674-d28a-4671-8a1f-2631e49492e1 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:41.891884 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 296/572] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:41.892799 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829665f-c9ae-422f-8c63-b62c5640add1 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:41.973281 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40aa674-d28a-4671-8a1f-2631e49492e1 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:42.000110 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829665f-c9ae-422f-8c63-b62c5640add1 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:42.009381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a0dfd-6490-4896-b7ba-519948c00fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:42.010570 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a0dfd-6490-4896-b7ba-519948c00fdf None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:42.010853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a0dfd-6490-4896-b7ba-519948c00fdf None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:42.011121 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a0dfd-6490-4896-b7ba-519948c00fdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:42.022671 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a0dfd-6490-4896-b7ba-519948c00fdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:42.024176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a0dfd-6490-4896-b7ba-519948c00fdf None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:42.038671 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5fde4-bf9b-45a3-b17e-86b7097958a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:42.039651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5fde4-bf9b-45a3-b17e-86b7097958a2 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:42.040089 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5fde4-bf9b-45a3-b17e-86b7097958a2 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:42.040359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5fde4-bf9b-45a3-b17e-86b7097958a2 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:42.041111 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5fde4-bf9b-45a3-b17e-86b7097958a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:42.041414 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5fde4-bf9b-45a3-b17e-86b7097958a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-cb9a0dfd-6490-4896-b7ba-519948c00fdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:42.041653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:42.042908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:42.044345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:42.044345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:42.044345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:42.044345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:42.044604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5fde4-bf9b-45a3-b17e-86b7097958a2 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:42.044677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 277/573] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:42.044677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a0dfd-6490-4896-b7ba-519948c00fdf None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:42.055285 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 297/574] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:42.056131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5fde4-bf9b-45a3-b17e-86b7097958a2 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:42.147971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a0dfd-6490-4896-b7ba-519948c00fdf None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:42.149607 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5fde4-bf9b-45a3-b17e-86b7097958a2 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:42.182116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6324c-b8f3-4f97-a6f4-02845094e01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:42.182443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ec552-3fd8-4ce0-bf66-d621581fdb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:42.182912 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6324c-b8f3-4f97-a6f4-02845094e01e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:42.183213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6324c-b8f3-4f97-a6f4-02845094e01e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:42.183315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ec552-3fd8-4ce0-bf66-d621581fdb5d None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:42.183492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6324c-b8f3-4f97-a6f4-02845094e01e None admin] PATH_INFO: `/v3/roles/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:42.183589 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ec552-3fd8-4ce0-bf66-d621581fdb5d None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:42.183781 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ec552-3fd8-4ce0-bf66-d621581fdb5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:42.194253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ec552-3fd8-4ce0-bf66-d621581fdb5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:42.195302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6324c-b8f3-4f97-a6f4-02845094e01e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-381ec552-3fd8-4ce0-bf66-d621581fdb5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:42.201923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:42.202928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:42.204349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:42.204349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:42.204349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:42.204349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:42.204349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:42.204349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:42.204349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:42.204349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:42.204349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:42.204349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:42.204349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:42.204349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-9dd6324c-b8f3-4f97-a6f4-02845094e01e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:42.204991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:42.205854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:42.206678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:42.206678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:42.206678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:42.206678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:42.206678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:42.206678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:42.206678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:42.206678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:42.206678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:42.206678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:42.206678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:42.206678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:42.207352 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 298/575] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:42.207352 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ec552-3fd8-4ce0-bf66-d621581fdb5d None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:42.213324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 278/576] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:42.213324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6324c-b8f3-4f97-a6f4-02845094e01e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:42.308977 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ec552-3fd8-4ce0-bf66-d621581fdb5d None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:42.317827 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6324c-b8f3-4f97-a6f4-02845094e01e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:42.340455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf65d1-fdec-467f-9716-d3271635fe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:42.341300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf65d1-fdec-467f-9716-d3271635fe64 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:42.341587 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf65d1-fdec-467f-9716-d3271635fe64 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:42.341865 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf65d1-fdec-467f-9716-d3271635fe64 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:42.342579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf65d1-fdec-467f-9716-d3271635fe64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:42.342881 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf65d1-fdec-467f-9716-d3271635fe64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:42.344395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf65d1-fdec-467f-9716-d3271635fe64 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:42.350970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4aeacd-cef6-420f-9a75-fe798cc2f55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:42.351805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4aeacd-cef6-420f-9a75-fe798cc2f55f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:42.352092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4aeacd-cef6-420f-9a75-fe798cc2f55f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:42.352369 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4aeacd-cef6-420f-9a75-fe798cc2f55f None admin] PATH_INFO: `/v3/users/nova` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:42.357620 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 299/577] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:42.360119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf65d1-fdec-467f-9716-d3271635fe64 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:42.365621 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4aeacd-cef6-420f-9a75-fe798cc2f55f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:42.367107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4aeacd-cef6-420f-9a75-fe798cc2f55f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-4f4aeacd-cef6-420f-9a75-fe798cc2f55f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:42.381826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:42.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:42.384152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:42.384152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:42.384152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:42.384152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:42.384152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:42.384152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:42.384152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:42.384152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:42.384152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 18:16:42.384152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:42.384152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 279/578] 10.222.0.25 () {56 vars in 1093 bytes} [Mon May 9 18:16:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:42.384152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4aeacd-cef6-420f-9a75-fe798cc2f55f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:42.461213 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf65d1-fdec-467f-9716-d3271635fe64 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:42.481906 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4aeacd-cef6-420f-9a75-fe798cc2f55f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:42.494096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2075d1-e471-43b8-97c8-2fe934016ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:42.494757 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2075d1-e471-43b8-97c8-2fe934016ca1 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:42.495033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2075d1-e471-43b8-97c8-2fe934016ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:42.495341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2075d1-e471-43b8-97c8-2fe934016ca1 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:42.496067 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2075d1-e471-43b8-97c8-2fe934016ca1 None admin] RBAC: Adding query filter params (name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:42.496369 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2075d1-e471-43b8-97c8-2fe934016ca1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:42.497716 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2075d1-e471-43b8-97c8-2fe934016ca1 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:42.505808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 300/579] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:42.507053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2075d1-e471-43b8-97c8-2fe934016ca1 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:42.520353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5c0a0-f97d-48b1-85e8-8feec3d535db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:42.521032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5c0a0-f97d-48b1-85e8-8feec3d535db None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:42.521316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5c0a0-f97d-48b1-85e8-8feec3d535db None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:42.521593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5c0a0-f97d-48b1-85e8-8feec3d535db None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:42.522342 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5c0a0-f97d-48b1-85e8-8feec3d535db None admin] RBAC: Adding query filter params (name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:42.522649 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5c0a0-f97d-48b1-85e8-8feec3d535db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:42.524026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5c0a0-f97d-48b1-85e8-8feec3d535db None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:42.536956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 280/580] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:42.536956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5c0a0-f97d-48b1-85e8-8feec3d535db None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:42.607681 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2075d1-e471-43b8-97c8-2fe934016ca1 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:42.634551 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5c0a0-f97d-48b1-85e8-8feec3d535db None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:42.639726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1153f-8a5d-45cf-a041-68c09440aedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:42.640519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1153f-8a5d-45cf-a041-68c09440aedb None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:42.640796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1153f-8a5d-45cf-a041-68c09440aedb None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:42.641070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1153f-8a5d-45cf-a041-68c09440aedb None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:42.648218 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1153f-8a5d-45cf-a041-68c09440aedb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:42.649648 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1153f-8a5d-45cf-a041-68c09440aedb None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-1ee1153f-8a5d-45cf-a041-68c09440aedb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:42.664333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:42.665230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:42.666109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:42.666109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:42.666109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:42.666109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:42.666109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:42.666109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:42.666109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:42.666109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:42.666109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 301/581] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:42.666597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1153f-8a5d-45cf-a041-68c09440aedb None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:42.671756 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8ccbc-4689-469d-a154-9be3a44db7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:42.672478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8ccbc-4689-469d-a154-9be3a44db7d0 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:42.672762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8ccbc-4689-469d-a154-9be3a44db7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:42.673031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8ccbc-4689-469d-a154-9be3a44db7d0 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:42.673888 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8ccbc-4689-469d-a154-9be3a44db7d0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:42.674193 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8ccbc-4689-469d-a154-9be3a44db7d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:42.675630 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8ccbc-4689-469d-a154-9be3a44db7d0 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:42.693349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 281/582] 10.222.0.25 () {56 vars in 1133 bytes} [Mon May 9 18:16:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:42.694185 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8ccbc-4689-469d-a154-9be3a44db7d0 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:42.762592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1153f-8a5d-45cf-a041-68c09440aedb None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:42.790126 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8ccbc-4689-469d-a154-9be3a44db7d0 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:42.795791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fd8dc-09a8-4bd7-b6f9-09d07fd28302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:42.796570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fd8dc-09a8-4bd7-b6f9-09d07fd28302 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:42.796848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fd8dc-09a8-4bd7-b6f9-09d07fd28302 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:42.797125 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fd8dc-09a8-4bd7-b6f9-09d07fd28302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:42.806639 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5fd8dc-09a8-4bd7-b6f9-09d07fd28302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:42.808103 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5fd8dc-09a8-4bd7-b6f9-09d07fd28302 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:42.822098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79ba9a-7460-4866-b31e-2ef90172d79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:42.822878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79ba9a-7460-4866-b31e-2ef90172d79e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:42.823160 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79ba9a-7460-4866-b31e-2ef90172d79e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:42.823470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79ba9a-7460-4866-b31e-2ef90172d79e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-dd5fd8dc-09a8-4bd7-b6f9-09d07fd28302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:42.824592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:42.825454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:42.826378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:42.826378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:42.826378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:42.826378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:42.826378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 302/583] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:42.826651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fd8dc-09a8-4bd7-b6f9-09d07fd28302 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:42.838817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79ba9a-7460-4866-b31e-2ef90172d79e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:42.840331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79ba9a-7460-4866-b31e-2ef90172d79e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-3a79ba9a-7460-4866-b31e-2ef90172d79e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:42.857606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:42.858529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:42.864215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:42.864215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:42.864215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:42.864215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:42.864215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 282/584] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:42.864215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79ba9a-7460-4866-b31e-2ef90172d79e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:42.920156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fd8dc-09a8-4bd7-b6f9-09d07fd28302 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:42.951957 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b7ca0-5dab-452b-8004-3e5e0df7564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:42.952609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b7ca0-5dab-452b-8004-3e5e0df7564a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:42.952885 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b7ca0-5dab-452b-8004-3e5e0df7564a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:42.953159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b7ca0-5dab-452b-8004-3e5e0df7564a None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:42.953921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b7ca0-5dab-452b-8004-3e5e0df7564a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:42.954223 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b7ca0-5dab-452b-8004-3e5e0df7564a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:42.956212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b7ca0-5dab-452b-8004-3e5e0df7564a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:42.965252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79ba9a-7460-4866-b31e-2ef90172d79e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:42.968048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 303/585] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:42.968746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b7ca0-5dab-452b-8004-3e5e0df7564a None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:43.010573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c60099-1ae5-44ee-b5fe-15bc7fdc0e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:43.011409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c60099-1ae5-44ee-b5fe-15bc7fdc0e10 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:43.011693 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c60099-1ae5-44ee-b5fe-15bc7fdc0e10 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:43.011966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c60099-1ae5-44ee-b5fe-15bc7fdc0e10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:43.022350 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c60099-1ae5-44ee-b5fe-15bc7fdc0e10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:43.023857 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c60099-1ae5-44ee-b5fe-15bc7fdc0e10 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-04c60099-1ae5-44ee-b5fe-15bc7fdc0e10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:43.041145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:43.042077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:43.047792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:43.047792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:43.047792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:43.047792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:43.047792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 283/586] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:43.047792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c60099-1ae5-44ee-b5fe-15bc7fdc0e10 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:43.068522 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b7ca0-5dab-452b-8004-3e5e0df7564a None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:43.101298 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba81d1-aaae-49e0-be7b-0e84f89b6610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:43.101959 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba81d1-aaae-49e0-be7b-0e84f89b6610 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:43.102241 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba81d1-aaae-49e0-be7b-0e84f89b6610 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:43.102518 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba81d1-aaae-49e0-be7b-0e84f89b6610 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:43.103263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba81d1-aaae-49e0-be7b-0e84f89b6610 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:43.103568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba81d1-aaae-49e0-be7b-0e84f89b6610 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:43.105058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba81d1-aaae-49e0-be7b-0e84f89b6610 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:43.121079 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 304/587] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:43.121079 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba81d1-aaae-49e0-be7b-0e84f89b6610 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:43.144657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c60099-1ae5-44ee-b5fe-15bc7fdc0e10 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:43.177226 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932281ff-aed4-4f8d-a7d9-56ac3714b1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:43.178128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932281ff-aed4-4f8d-a7d9-56ac3714b1a2 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:43.178595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932281ff-aed4-4f8d-a7d9-56ac3714b1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:43.178872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932281ff-aed4-4f8d-a7d9-56ac3714b1a2 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:43.179617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932281ff-aed4-4f8d-a7d9-56ac3714b1a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:43.179925 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932281ff-aed4-4f8d-a7d9-56ac3714b1a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:43.181275 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932281ff-aed4-4f8d-a7d9-56ac3714b1a2 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:43.192587 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 284/588] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:43.193398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932281ff-aed4-4f8d-a7d9-56ac3714b1a2 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:43.218662 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba81d1-aaae-49e0-be7b-0e84f89b6610 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:43.252743 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ee679-cfc2-4eda-bb40-daf4d0aac451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:43.253700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ee679-cfc2-4eda-bb40-daf4d0aac451 None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:43.254042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ee679-cfc2-4eda-bb40-daf4d0aac451 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:43.254319 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ee679-cfc2-4eda-bb40-daf4d0aac451 None admin] PATH_INFO: `/v3/projects/4849b58351c74caf92a584686c74e397/users/77cf284e81ba4042b66c0bc189f1ae2b/roles/7b2e521b45dc4a98adad996ca0027de6` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:43.268759 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ee679-cfc2-4eda-bb40-daf4d0aac451 None admin] RBAC: Authorizing `identity:create_grant(project_id=4849b58351c74caf92a584686c74e397, user_id=77cf284e81ba4042b66c0bc189f1ae2b, role_id=7b2e521b45dc4a98adad996ca0027de6)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:43.270166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ee679-cfc2-4eda-bb40-daf4d0aac451 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:43.295209 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 305/589] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:16:43 2022] PUT /identity/v3/projects/4849b58351c74caf92a584686c74e397/users/77cf284e81ba4042b66c0bc189f1ae2b/roles/7b2e521b45dc4a98adad996ca0027de6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:16:43.295209 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ee679-cfc2-4eda-bb40-daf4d0aac451 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:43.307024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932281ff-aed4-4f8d-a7d9-56ac3714b1a2 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:43.354770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd75a1-51bc-48bb-8e8b-ba5b425d979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:43.356747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd75a1-51bc-48bb-8e8b-ba5b425d979d None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:43.357029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd75a1-51bc-48bb-8e8b-ba5b425d979d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:43.357304 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd75a1-51bc-48bb-8e8b-ba5b425d979d None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:43.358038 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd75a1-51bc-48bb-8e8b-ba5b425d979d None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:43.358343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd75a1-51bc-48bb-8e8b-ba5b425d979d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:43.359777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd75a1-51bc-48bb-8e8b-ba5b425d979d None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:43.379420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 285/590] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:43.379420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd75a1-51bc-48bb-8e8b-ba5b425d979d None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:43.410903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ee679-cfc2-4eda-bb40-daf4d0aac451 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:43.448587 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9723cf-1a92-451c-8e96-09e5af6ed223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:43.449458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9723cf-1a92-451c-8e96-09e5af6ed223 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:43.449678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9723cf-1a92-451c-8e96-09e5af6ed223 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:43.449952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9723cf-1a92-451c-8e96-09e5af6ed223 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:43.458211 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9723cf-1a92-451c-8e96-09e5af6ed223 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:43.459769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9723cf-1a92-451c-8e96-09e5af6ed223 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-7b9723cf-1a92-451c-8e96-09e5af6ed223 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:43.477505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:43.478485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:43.479850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:43.479850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:43.479850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:43.479850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:43.479850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:43.479850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:43.479850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:43.479850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:43.479850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 18:16:43.479850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:43.479850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 306/591] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:43.480660 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9723cf-1a92-451c-8e96-09e5af6ed223 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:43.496243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd75a1-51bc-48bb-8e8b-ba5b425d979d None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:43.540532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c27167-02d8-4ac5-8543-7fb797db0e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:43.541338 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c27167-02d8-4ac5-8543-7fb797db0e39 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:43.541620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c27167-02d8-4ac5-8543-7fb797db0e39 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:43.542329 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c27167-02d8-4ac5-8543-7fb797db0e39 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:43.551093 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c27167-02d8-4ac5-8543-7fb797db0e39 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:43.552578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c27167-02d8-4ac5-8543-7fb797db0e39 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-43c27167-02d8-4ac5-8543-7fb797db0e39 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:43.567981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:43.568945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:43.569861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:43.569861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:43.569861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:43.569861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:43.569861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:43.569861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:43.569861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:43.569861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:43.569861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 18:16:43.569861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:43.569861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 286/592] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:43.570440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c27167-02d8-4ac5-8543-7fb797db0e39 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:43.595393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9723cf-1a92-451c-8e96-09e5af6ed223 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:43.641070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f553a9-e82f-44f2-b1c6-c22920e5bbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:43.641792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f553a9-e82f-44f2-b1c6-c22920e5bbfb None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:43.642073 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f553a9-e82f-44f2-b1c6-c22920e5bbfb None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:43.642348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f553a9-e82f-44f2-b1c6-c22920e5bbfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:43.651148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f553a9-e82f-44f2-b1c6-c22920e5bbfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:43.653118 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f553a9-e82f-44f2-b1c6-c22920e5bbfb None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-48f553a9-e82f-44f2-b1c6-c22920e5bbfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:43.672078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:43.674368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:43.675458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:43.675458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:43.675458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:43.675458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:43.675458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:43.675458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:43.675458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:43.675458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:43.675458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 307/593] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:43.675458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f553a9-e82f-44f2-b1c6-c22920e5bbfb None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:43.680633 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c27167-02d8-4ac5-8543-7fb797db0e39 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:43.715288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f225ad-fc3f-4550-9e21-ebbdfa6a4f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:43.715994 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f225ad-fc3f-4550-9e21-ebbdfa6a4f6d None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:43.716277 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f225ad-fc3f-4550-9e21-ebbdfa6a4f6d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:43.716553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f225ad-fc3f-4550-9e21-ebbdfa6a4f6d None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:43.717389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f225ad-fc3f-4550-9e21-ebbdfa6a4f6d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:43.717697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f225ad-fc3f-4550-9e21-ebbdfa6a4f6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:43.719138 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f225ad-fc3f-4550-9e21-ebbdfa6a4f6d None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:43.737791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 287/594] 10.222.0.25 () {56 vars in 1137 bytes} [Mon May 9 18:16:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:43.738656 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f225ad-fc3f-4550-9e21-ebbdfa6a4f6d None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:43.789177 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f553a9-e82f-44f2-b1c6-c22920e5bbfb None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:43.823023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45800d3d-c866-4b00-aa56-c25fdc8f6c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:43.823721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45800d3d-c866-4b00-aa56-c25fdc8f6c50 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:43.824008 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45800d3d-c866-4b00-aa56-c25fdc8f6c50 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:43.824282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45800d3d-c866-4b00-aa56-c25fdc8f6c50 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:43.825163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45800d3d-c866-4b00-aa56-c25fdc8f6c50 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:43.825467 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45800d3d-c866-4b00-aa56-c25fdc8f6c50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:43.826942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45800d3d-c866-4b00-aa56-c25fdc8f6c50 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:43.837081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f225ad-fc3f-4550-9e21-ebbdfa6a4f6d None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:43.845191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 308/595] 10.222.0.25 () {56 vars in 1139 bytes} [Mon May 9 18:16:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:43.846565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45800d3d-c866-4b00-aa56-c25fdc8f6c50 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:43.875086 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cd0b8-bd5d-4dd1-be4a-7099cf165035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:43.875785 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cd0b8-bd5d-4dd1-be4a-7099cf165035 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:43.876065 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cd0b8-bd5d-4dd1-be4a-7099cf165035 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:43.876336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cd0b8-bd5d-4dd1-be4a-7099cf165035 None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:43.877147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cd0b8-bd5d-4dd1-be4a-7099cf165035 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:43.877716 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cd0b8-bd5d-4dd1-be4a-7099cf165035 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:43.879151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cd0b8-bd5d-4dd1-be4a-7099cf165035 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:43.890842 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 288/596] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:43.891839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cd0b8-bd5d-4dd1-be4a-7099cf165035 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:43.951752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45800d3d-c866-4b00-aa56-c25fdc8f6c50 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:43.995201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d0699-8972-4fc0-a42e-d474060cfaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:43.995992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d0699-8972-4fc0-a42e-d474060cfaf3 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:43.996273 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d0699-8972-4fc0-a42e-d474060cfaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:43.996546 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d0699-8972-4fc0-a42e-d474060cfaf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:43.997373 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cd0b8-bd5d-4dd1-be4a-7099cf165035 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:44.009252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d0699-8972-4fc0-a42e-d474060cfaf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:44.010857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d0699-8972-4fc0-a42e-d474060cfaf3 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:44.031282 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042c73c-432d-4237-b966-ca55a306010f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:44.032797 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042c73c-432d-4237-b966-ca55a306010f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-345d0699-8972-4fc0-a42e-d474060cfaf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:44.032973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:44.034341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:44.035256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:44.035256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:44.035256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:44.035256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:44.036182 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042c73c-432d-4237-b966-ca55a306010f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:44.036702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042c73c-432d-4237-b966-ca55a306010f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:44.039456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 309/597] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:44.040611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d0699-8972-4fc0-a42e-d474060cfaf3 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:44.049612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042c73c-432d-4237-b966-ca55a306010f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:44.051405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042c73c-432d-4237-b966-ca55a306010f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-d042c73c-432d-4237-b966-ca55a306010f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:44.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:44.069930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:44.070831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:44.070831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:44.070831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:44.070831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:44.073096 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 289/598] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:44.074191 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042c73c-432d-4237-b966-ca55a306010f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:44.147529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d0699-8972-4fc0-a42e-d474060cfaf3 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:44.173592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042c73c-432d-4237-b966-ca55a306010f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:44.182569 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322366f1-c37d-4f8e-99a5-6095feb372ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:44.183490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322366f1-c37d-4f8e-99a5-6095feb372ce None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:44.183775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322366f1-c37d-4f8e-99a5-6095feb372ce None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:44.184047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322366f1-c37d-4f8e-99a5-6095feb372ce None admin] PATH_INFO: `/v3/projects/4849b58351c74caf92a584686c74e397/users/f769ee9f31354e5296645fb1e3bd84b6/roles/7b2e521b45dc4a98adad996ca0027de6` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:44.198370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322366f1-c37d-4f8e-99a5-6095feb372ce None admin] RBAC: Authorizing `identity:create_grant(project_id=4849b58351c74caf92a584686c74e397, user_id=f769ee9f31354e5296645fb1e3bd84b6, role_id=7b2e521b45dc4a98adad996ca0027de6)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:44.199842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322366f1-c37d-4f8e-99a5-6095feb372ce None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:44.207044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d09ae-2e1b-45f7-a1d4-b85c3a88348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:44.207816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d09ae-2e1b-45f7-a1d4-b85c3a88348f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:44.208103 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d09ae-2e1b-45f7-a1d4-b85c3a88348f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:44.208376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d09ae-2e1b-45f7-a1d4-b85c3a88348f None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:44.209103 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d09ae-2e1b-45f7-a1d4-b85c3a88348f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:44.209401 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d09ae-2e1b-45f7-a1d4-b85c3a88348f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:44.210745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d09ae-2e1b-45f7-a1d4-b85c3a88348f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:44.223483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 310/599] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:16:44 2022] PUT /identity/v3/projects/4849b58351c74caf92a584686c74e397/users/f769ee9f31354e5296645fb1e3bd84b6/roles/7b2e521b45dc4a98adad996ca0027de6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:16:44.223483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322366f1-c37d-4f8e-99a5-6095feb372ce None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:44.223764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 290/600] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:44.224943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcf41e-ec34-45db-860b-4c6edb053e3e None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:44.225231 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcf41e-ec34-45db-860b-4c6edb053e3e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:44.225507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcf41e-ec34-45db-860b-4c6edb053e3e None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:44.227058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 291/601] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:44.233612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcf41e-ec34-45db-860b-4c6edb053e3e None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:44.342054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcf41e-ec34-45db-860b-4c6edb053e3e None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:44.357637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322366f1-c37d-4f8e-99a5-6095feb372ce None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:44.377791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2b28d-1aa4-4b05-9e3e-6bb23772cd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:44.378599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2b28d-1aa4-4b05-9e3e-6bb23772cd9e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:44.378889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2b28d-1aa4-4b05-9e3e-6bb23772cd9e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:44.379166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2b28d-1aa4-4b05-9e3e-6bb23772cd9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:44.386434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d2b28d-1aa4-4b05-9e3e-6bb23772cd9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:44.387935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d2b28d-1aa4-4b05-9e3e-6bb23772cd9e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-66d2b28d-1aa4-4b05-9e3e-6bb23772cd9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:44.402854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:44.403874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:44.404752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:44.404752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:44.404752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:44.404752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:44.404752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:44.404752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:44.404752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:44.404752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:44.406045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e9d6a-a8c7-4752-947e-23d4cf2d0722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:44.406944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e9d6a-a8c7-4752-947e-23d4cf2d0722 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:44.407420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e9d6a-a8c7-4752-947e-23d4cf2d0722 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:44.407843 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e9d6a-a8c7-4752-947e-23d4cf2d0722 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:44.409334 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e9d6a-a8c7-4752-947e-23d4cf2d0722 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:44.409470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 292/602] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:44.409927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e9d6a-a8c7-4752-947e-23d4cf2d0722 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:44.410735 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488f840-3978-4b7f-bb5b-9a5537f713dc None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:44.411029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488f840-3978-4b7f-bb5b-9a5537f713dc None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:44.411339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488f840-3978-4b7f-bb5b-9a5537f713dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:44.412047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e9d6a-a8c7-4752-947e-23d4cf2d0722 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:44.425054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 311/603] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:44.426141 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e9d6a-a8c7-4752-947e-23d4cf2d0722 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:44.431515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-b488f840-3978-4b7f-bb5b-9a5537f713dc None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:44.433675 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b488f840-3978-4b7f-bb5b-9a5537f713dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:44.527324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 293/604] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:44.528281 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488f840-3978-4b7f-bb5b-9a5537f713dc None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:44.540004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e9d6a-a8c7-4752-947e-23d4cf2d0722 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:44.576056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a499b-d3b8-4192-a221-afa2a146544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:44.576872 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a499b-d3b8-4192-a221-afa2a146544e None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:44.577257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a499b-d3b8-4192-a221-afa2a146544e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:44.577639 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a499b-d3b8-4192-a221-afa2a146544e None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:44.578574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a499b-d3b8-4192-a221-afa2a146544e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:44.578981 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a499b-d3b8-4192-a221-afa2a146544e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:44.580569 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a499b-d3b8-4192-a221-afa2a146544e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:44.593464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 312/605] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:44.600323 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a499b-d3b8-4192-a221-afa2a146544e None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:44.649077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488f840-3978-4b7f-bb5b-9a5537f713dc None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:44.684240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068e7da-41be-4ee6-88d8-97a85664bd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:44.685265 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068e7da-41be-4ee6-88d8-97a85664bd66 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:44.685679 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068e7da-41be-4ee6-88d8-97a85664bd66 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:44.686038 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068e7da-41be-4ee6-88d8-97a85664bd66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:44.693394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068e7da-41be-4ee6-88d8-97a85664bd66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:44.694975 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068e7da-41be-4ee6-88d8-97a85664bd66 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:44.705498 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a499b-d3b8-4192-a221-afa2a146544e None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-8068e7da-41be-4ee6-88d8-97a85664bd66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:44.709424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:44.710482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:44.711524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:44.711524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:44.711524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:44.711524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:44.711524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:44.711524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:44.711524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:44.711524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:44.712637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 294/606] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:44.713492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068e7da-41be-4ee6-88d8-97a85664bd66 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:44.743852 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4f093-fc5e-4b33-a4c6-ed6e46afd738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:44.745053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4f093-fc5e-4b33-a4c6-ed6e46afd738 None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:44.745519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4f093-fc5e-4b33-a4c6-ed6e46afd738 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:44.746031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4f093-fc5e-4b33-a4c6-ed6e46afd738 None admin] PATH_INFO: `/v3/projects/4849b58351c74caf92a584686c74e397/users/4bda352a10f241ca96d5308a55238d57/roles/7b2e521b45dc4a98adad996ca0027de6` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:44.761270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac4f093-fc5e-4b33-a4c6-ed6e46afd738 None admin] RBAC: Authorizing `identity:create_grant(project_id=4849b58351c74caf92a584686c74e397, user_id=4bda352a10f241ca96d5308a55238d57, role_id=7b2e521b45dc4a98adad996ca0027de6)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:44.763016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac4f093-fc5e-4b33-a4c6-ed6e46afd738 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:44.778547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 313/607] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:16:44 2022] PUT /identity/v3/projects/4849b58351c74caf92a584686c74e397/users/4bda352a10f241ca96d5308a55238d57/roles/7b2e521b45dc4a98adad996ca0027de6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:16:44.779871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4f093-fc5e-4b33-a4c6-ed6e46afd738 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:44.852996 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068e7da-41be-4ee6-88d8-97a85664bd66 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:44.894280 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931c1bc-8c50-4a6e-b3b6-753084d1f318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:44.895163 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931c1bc-8c50-4a6e-b3b6-753084d1f318 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:44.895470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931c1bc-8c50-4a6e-b3b6-753084d1f318 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:44.895746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931c1bc-8c50-4a6e-b3b6-753084d1f318 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:44.921098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931c1bc-8c50-4a6e-b3b6-753084d1f318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-6931c1bc-8c50-4a6e-b3b6-753084d1f318 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:44.933409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:44.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:44.935383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:44.935383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:44.935383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:44.935383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:44.935383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:44.935383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:44.935383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:44.935383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:44.935383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:44.935383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:44.935383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:44.935383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:44.937817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 295/608] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:44.943055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931c1bc-8c50-4a6e-b3b6-753084d1f318 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:44.961301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4f093-fc5e-4b33-a4c6-ed6e46afd738 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:45.049888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01fa62-98d4-4d7c-83c7-9ba6e11617c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:45.050556 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01fa62-98d4-4d7c-83c7-9ba6e11617c4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:45.050838 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01fa62-98d4-4d7c-83c7-9ba6e11617c4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:45.051110 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01fa62-98d4-4d7c-83c7-9ba6e11617c4 None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:45.051926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01fa62-98d4-4d7c-83c7-9ba6e11617c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:45.052229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01fa62-98d4-4d7c-83c7-9ba6e11617c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:45.053663 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01fa62-98d4-4d7c-83c7-9ba6e11617c4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:45.073692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 314/609] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:45.079332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01fa62-98d4-4d7c-83c7-9ba6e11617c4 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:45.109113 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931c1bc-8c50-4a6e-b3b6-753084d1f318 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:45.174290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c606ff-6ab7-4ba0-b0ca-ff8c45797553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:45.174966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c606ff-6ab7-4ba0-b0ca-ff8c45797553 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:45.175265 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c606ff-6ab7-4ba0-b0ca-ff8c45797553 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:45.175537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c606ff-6ab7-4ba0-b0ca-ff8c45797553 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:45.176284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c606ff-6ab7-4ba0-b0ca-ff8c45797553 None admin] RBAC: Adding query filter params (name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:45.176583 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c606ff-6ab7-4ba0-b0ca-ff8c45797553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:45.177942 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c606ff-6ab7-4ba0-b0ca-ff8c45797553 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:45.193612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 296/610] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:45.203009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c606ff-6ab7-4ba0-b0ca-ff8c45797553 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:45.247819 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01fa62-98d4-4d7c-83c7-9ba6e11617c4 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:45.302880 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c606ff-6ab7-4ba0-b0ca-ff8c45797553 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:45.306695 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70955dd-b5df-4478-a9b9-c0cd35a733af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:45.307612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70955dd-b5df-4478-a9b9-c0cd35a733af None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:45.307894 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70955dd-b5df-4478-a9b9-c0cd35a733af None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:45.308173 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70955dd-b5df-4478-a9b9-c0cd35a733af None admin] PATH_INFO: `/v3/projects/4849b58351c74caf92a584686c74e397/users/6b2b6318776b422d9701351ba6b4ee27/roles/7b2e521b45dc4a98adad996ca0027de6` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:45.336923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b9125-56d6-4494-86e0-64abf7e101f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:45.337713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b9125-56d6-4494-86e0-64abf7e101f2 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:45.337992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b9125-56d6-4494-86e0-64abf7e101f2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:45.338265 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b9125-56d6-4494-86e0-64abf7e101f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:45.344500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70955dd-b5df-4478-a9b9-c0cd35a733af None admin] RBAC: Authorizing `identity:create_grant(project_id=4849b58351c74caf92a584686c74e397, user_id=6b2b6318776b422d9701351ba6b4ee27, role_id=7b2e521b45dc4a98adad996ca0027de6)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:45.346219 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70955dd-b5df-4478-a9b9-c0cd35a733af None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:45.352924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b9125-56d6-4494-86e0-64abf7e101f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:45.352924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b9125-56d6-4494-86e0-64abf7e101f2 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-cf2b9125-56d6-4494-86e0-64abf7e101f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:45.368453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:45.369411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:45.370324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:45.370324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:45.370324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:45.370324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:45.370324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 297/611] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:45.373690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 315/612] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:16:45 2022] PUT /identity/v3/projects/4849b58351c74caf92a584686c74e397/users/6b2b6318776b422d9701351ba6b4ee27/roles/7b2e521b45dc4a98adad996ca0027de6 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:16:45.375356 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b9125-56d6-4494-86e0-64abf7e101f2 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:45.486137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b9125-56d6-4494-86e0-64abf7e101f2 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:45.526201 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d2f45-14ee-46ee-844d-7cffb713f4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:45.526875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d2f45-14ee-46ee-844d-7cffb713f4a7 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:45.527155 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d2f45-14ee-46ee-844d-7cffb713f4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:45.527464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d2f45-14ee-46ee-844d-7cffb713f4a7 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:45.528179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d2f45-14ee-46ee-844d-7cffb713f4a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:45.528476 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d2f45-14ee-46ee-844d-7cffb713f4a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:45.529824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d2f45-14ee-46ee-844d-7cffb713f4a7 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:45.543133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 298/613] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:45.548768 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70955dd-b5df-4478-a9b9-c0cd35a733af None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:45.696779 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70955dd-b5df-4478-a9b9-c0cd35a733af None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:45.734767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd7fce-5095-46d0-bd2e-e2f0e65f6148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:45.735626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd7fce-5095-46d0-bd2e-e2f0e65f6148 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:45.736023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd7fce-5095-46d0-bd2e-e2f0e65f6148 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:45.736883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd7fce-5095-46d0-bd2e-e2f0e65f6148 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:45.746772 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd7fce-5095-46d0-bd2e-e2f0e65f6148 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:45.748242 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd7fce-5095-46d0-bd2e-e2f0e65f6148 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-3ddd7fce-5095-46d0-bd2e-e2f0e65f6148 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:45.764537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:45.765496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:45.766494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:45.766494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:45.766494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:45.766494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:45.766494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:45.766494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:45.766494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:45.766494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:45.766494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 18:16:45.766494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:45.766494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 316/614] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:16:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:45.771562 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d2f45-14ee-46ee-844d-7cffb713f4a7 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:45.773280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f561e94-b631-417c-bc0f-4a6eaace83fb None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:45.773570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f561e94-b631-417c-bc0f-4a6eaace83fb None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:45.773851 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f561e94-b631-417c-bc0f-4a6eaace83fb None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:45.775163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 317/615] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:16:45.785703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c08046-6483-4a2d-a911-240e2f9feb93 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:45.785969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c08046-6483-4a2d-a911-240e2f9feb93 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:45.786254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c08046-6483-4a2d-a911-240e2f9feb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:45.816201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-99c08046-6483-4a2d-a911-240e2f9feb93 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:45.817714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-99c08046-6483-4a2d-a911-240e2f9feb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:45.871448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d2f45-14ee-46ee-844d-7cffb713f4a7 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:45.909674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770a5a3-6068-4838-b130-3fdb151e6107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:45.910350 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770a5a3-6068-4838-b130-3fdb151e6107 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:45.910626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770a5a3-6068-4838-b130-3fdb151e6107 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:45.910896 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770a5a3-6068-4838-b130-3fdb151e6107 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:45.911750 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770a5a3-6068-4838-b130-3fdb151e6107 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:45.912057 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770a5a3-6068-4838-b130-3fdb151e6107 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:45.913695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770a5a3-6068-4838-b130-3fdb151e6107 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:45.945410 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 299/616] 10.222.0.25 () {56 vars in 1135 bytes} [Mon May 9 18:16:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:45.950752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770a5a3-6068-4838-b130-3fdb151e6107 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:45.950889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 318/617] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 18:16:46.002751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c08046-6483-4a2d-a911-240e2f9feb93 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:46.124704 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770a5a3-6068-4838-b130-3fdb151e6107 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:46.160036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd1754-989e-4b66-818b-52f6889c5734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:46.160844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd1754-989e-4b66-818b-52f6889c5734 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:46.161120 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd1754-989e-4b66-818b-52f6889c5734 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:46.161387 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd1754-989e-4b66-818b-52f6889c5734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:46.171741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd1754-989e-4b66-818b-52f6889c5734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:46.173229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd1754-989e-4b66-818b-52f6889c5734 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-1bcd1754-989e-4b66-818b-52f6889c5734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:46.191286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:46.192211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:46.193081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:46.193081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:46.193081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:46.193081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:46.193299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c08046-6483-4a2d-a911-240e2f9feb93 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:46.193381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 300/618] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:46.200612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd1754-989e-4b66-818b-52f6889c5734 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:46.292124 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08b810-bb59-4042-971b-b273aabc890c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:46.292888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08b810-bb59-4042-971b-b273aabc890c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:46.293174 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08b810-bb59-4042-971b-b273aabc890c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:46.293454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08b810-bb59-4042-971b-b273aabc890c None admin] PATH_INFO: `/v3/roles/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:46.318627 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd1754-989e-4b66-818b-52f6889c5734 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:46.326292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08b810-bb59-4042-971b-b273aabc890c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-cc08b810-bb59-4042-971b-b273aabc890c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:46.343961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:46.346159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:46.348149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:46.348149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:46.348149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:46.348149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:46.348149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:46.348149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:46.348149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:46.348149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:46.348149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:46.348149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:46.348149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:46.348149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:46.348149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 319/619] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:46.355210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08b810-bb59-4042-971b-b273aabc890c None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:46.391881 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb5ef0-8b0c-4be4-8875-ca869582f721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:46.392608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb5ef0-8b0c-4be4-8875-ca869582f721 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:46.392898 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb5ef0-8b0c-4be4-8875-ca869582f721 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:46.393178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb5ef0-8b0c-4be4-8875-ca869582f721 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:46.393908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb5ef0-8b0c-4be4-8875-ca869582f721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:46.394217 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb5ef0-8b0c-4be4-8875-ca869582f721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:46.395614 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb5ef0-8b0c-4be4-8875-ca869582f721 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:46.426391 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 301/620] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:46.431896 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb5ef0-8b0c-4be4-8875-ca869582f721 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:46.460941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08b810-bb59-4042-971b-b273aabc890c None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:46.496714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c4e1c-0e13-432c-a3d5-a85d505f0078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:46.497432 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c4e1c-0e13-432c-a3d5-a85d505f0078 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:46.497714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c4e1c-0e13-432c-a3d5-a85d505f0078 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:46.497987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c4e1c-0e13-432c-a3d5-a85d505f0078 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:46.498741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c4e1c-0e13-432c-a3d5-a85d505f0078 None admin] RBAC: Adding query filter params (name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:46.499012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c4e1c-0e13-432c-a3d5-a85d505f0078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:46.500936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c4e1c-0e13-432c-a3d5-a85d505f0078 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:46.511392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 320/621] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:46.516535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c4e1c-0e13-432c-a3d5-a85d505f0078 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:46.628926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c4e1c-0e13-432c-a3d5-a85d505f0078 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:46.669929 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb5ef0-8b0c-4be4-8875-ca869582f721 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:46.680749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41500c58-16d1-4fee-a4f7-96344692480f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:46.681602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41500c58-16d1-4fee-a4f7-96344692480f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:46.681878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41500c58-16d1-4fee-a4f7-96344692480f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:46.682148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41500c58-16d1-4fee-a4f7-96344692480f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:46.694960 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41500c58-16d1-4fee-a4f7-96344692480f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:46.696593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41500c58-16d1-4fee-a4f7-96344692480f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-41500c58-16d1-4fee-a4f7-96344692480f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:46.728928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:46.731042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:46.733189 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:46.733189 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:46.733189 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:46.733189 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:46.743287 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 321/622] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:46.747202 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41500c58-16d1-4fee-a4f7-96344692480f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:46.784170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36046a60-741f-4eec-9186-7ba3e0f40563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:46.784990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36046a60-741f-4eec-9186-7ba3e0f40563 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:46.785276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36046a60-741f-4eec-9186-7ba3e0f40563 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:46.785555 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36046a60-741f-4eec-9186-7ba3e0f40563 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:46.820044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36046a60-741f-4eec-9186-7ba3e0f40563 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:46.821522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36046a60-741f-4eec-9186-7ba3e0f40563 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-36046a60-741f-4eec-9186-7ba3e0f40563 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:46.857532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:46.859382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:46.860471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:46.860471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:46.860471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:46.860471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:46.860471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:46.860471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:46.860471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:46.860471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:46.862919 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41500c58-16d1-4fee-a4f7-96344692480f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:46.863523 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 302/623] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:46.864927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cad10c-2954-48b3-8930-791a29e3b830 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:46.865212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cad10c-2954-48b3-8930-791a29e3b830 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:46.865487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cad10c-2954-48b3-8930-791a29e3b830 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:46.866809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 303/624] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:46.868498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cad10c-2954-48b3-8930-791a29e3b830 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:46.899775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dd3b5-64a1-46f8-ad7f-caa2bfa540ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:46.900354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dd3b5-64a1-46f8-ad7f-caa2bfa540ee None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:46.900630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dd3b5-64a1-46f8-ad7f-caa2bfa540ee None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:46.900902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dd3b5-64a1-46f8-ad7f-caa2bfa540ee None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:46.901667 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671dd3b5-64a1-46f8-ad7f-caa2bfa540ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:46.901968 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671dd3b5-64a1-46f8-ad7f-caa2bfa540ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:46.904848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671dd3b5-64a1-46f8-ad7f-caa2bfa540ee None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:46.927592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 322/625] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:46.929025 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233ad04-1d7d-4943-9e72-747c01aceb63 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:46.929312 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233ad04-1d7d-4943-9e72-747c01aceb63 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:46.931263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233ad04-1d7d-4943-9e72-747c01aceb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:46.972390 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2233ad04-1d7d-4943-9e72-747c01aceb63 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:46.973570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2233ad04-1d7d-4943-9e72-747c01aceb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:47.043115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 323/626] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:47.044003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233ad04-1d7d-4943-9e72-747c01aceb63 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:47.052845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cad10c-2954-48b3-8930-791a29e3b830 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:47.115495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4497619-9e3d-4c21-8723-2c6f565054cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:47.116180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4497619-9e3d-4c21-8723-2c6f565054cd None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:47.116465 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4497619-9e3d-4c21-8723-2c6f565054cd None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:47.116744 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4497619-9e3d-4c21-8723-2c6f565054cd None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:47.117531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4497619-9e3d-4c21-8723-2c6f565054cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:47.117843 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4497619-9e3d-4c21-8723-2c6f565054cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:47.129345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4497619-9e3d-4c21-8723-2c6f565054cd None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:47.144310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233ad04-1d7d-4943-9e72-747c01aceb63 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:47.172542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 304/627] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:47.173388 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4497619-9e3d-4c21-8723-2c6f565054cd None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:47.199969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526c9a8-ef89-4bd8-ace8-5b332c14b6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:47.201940 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526c9a8-ef89-4bd8-ace8-5b332c14b6ad None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:47.201940 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526c9a8-ef89-4bd8-ace8-5b332c14b6ad None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:47.201940 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526c9a8-ef89-4bd8-ace8-5b332c14b6ad None admin] PATH_INFO: `/v3/users/glance` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:47.211197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526c9a8-ef89-4bd8-ace8-5b332c14b6ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:47.213304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526c9a8-ef89-4bd8-ace8-5b332c14b6ad None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-4526c9a8-ef89-4bd8-ace8-5b332c14b6ad None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:47.237023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:47.238032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:47.238965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:47.238965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:47.238965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:47.238965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:47.238965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:47.238965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:47.238965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:47.238965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:47.238965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 18:16:47.238965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:47.238965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 324/628] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:47.240116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526c9a8-ef89-4bd8-ace8-5b332c14b6ad None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:47.335456 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4497619-9e3d-4c21-8723-2c6f565054cd None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:47.382730 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8194d-573f-41a9-9ed5-a796168426fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:47.383814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8194d-573f-41a9-9ed5-a796168426fe None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:47.384266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8194d-573f-41a9-9ed5-a796168426fe None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:47.384669 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526c9a8-ef89-4bd8-ace8-5b332c14b6ad None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:47.385179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8194d-573f-41a9-9ed5-a796168426fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:47.396490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8194d-573f-41a9-9ed5-a796168426fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-aef8194d-573f-41a9-9ed5-a796168426fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:47.404371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:47.405412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:47.406258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:47.406258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:47.406258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:47.406258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:47.406258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:47.406258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:47.406258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:47.406258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:47.406258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:47.406258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:47.406258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:47.406258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:47.409825 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 305/629] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:47.410806 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8194d-573f-41a9-9ed5-a796168426fe None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:47.421909 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7623acf-3974-4b66-837e-982e50784d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:47.423006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623acf-3974-4b66-837e-982e50784d2e None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:47.423325 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623acf-3974-4b66-837e-982e50784d2e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:47.423706 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623acf-3974-4b66-837e-982e50784d2e None admin] PATH_INFO: `/v3/projects/4849b58351c74caf92a584686c74e397/users/ec2db3aa2b7d41f89314fec960d898b7/roles/7b2e521b45dc4a98adad996ca0027de6` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:47.443083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7623acf-3974-4b66-837e-982e50784d2e None admin] RBAC: Authorizing `identity:create_grant(project_id=4849b58351c74caf92a584686c74e397, user_id=ec2db3aa2b7d41f89314fec960d898b7, role_id=7b2e521b45dc4a98adad996ca0027de6)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:47.445038 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7623acf-3974-4b66-837e-982e50784d2e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:47.466441 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 325/630] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:16:47 2022] PUT /identity/v3/projects/4849b58351c74caf92a584686c74e397/users/ec2db3aa2b7d41f89314fec960d898b7/roles/7b2e521b45dc4a98adad996ca0027de6 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:16:47.472465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7623acf-3974-4b66-837e-982e50784d2e None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:47.566566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8194d-573f-41a9-9ed5-a796168426fe None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:47.602316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c59b6-b9e1-432b-b5ab-5f959fcd017e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:47.603013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c59b6-b9e1-432b-b5ab-5f959fcd017e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:47.603332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c59b6-b9e1-432b-b5ab-5f959fcd017e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:47.603613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c59b6-b9e1-432b-b5ab-5f959fcd017e None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:47.604471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c59b6-b9e1-432b-b5ab-5f959fcd017e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:47.604853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c59b6-b9e1-432b-b5ab-5f959fcd017e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:47.606381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c59b6-b9e1-432b-b5ab-5f959fcd017e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:47.625484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 306/631] 10.222.0.25 () {56 vars in 1137 bytes} [Mon May 9 18:16:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:47.627582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0af7c1-c2bb-4898-850d-50154670f9cc None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:47.627889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0af7c1-c2bb-4898-850d-50154670f9cc None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:47.628167 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0af7c1-c2bb-4898-850d-50154670f9cc None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:47.633014 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7623acf-3974-4b66-837e-982e50784d2e None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:47.633725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 307/632] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:47.634672 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0af7c1-c2bb-4898-850d-50154670f9cc None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:47.675768 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd715b-6b40-4427-87f9-17f8fa63ff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:47.677653 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd715b-6b40-4427-87f9-17f8fa63ff70 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:47.678142 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd715b-6b40-4427-87f9-17f8fa63ff70 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:47.678661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd715b-6b40-4427-87f9-17f8fa63ff70 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:47.679681 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd715b-6b40-4427-87f9-17f8fa63ff70 None admin] RBAC: Adding query filter params (name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:47.680185 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd715b-6b40-4427-87f9-17f8fa63ff70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:47.682376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd715b-6b40-4427-87f9-17f8fa63ff70 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:47.694220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 326/633] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:47.696045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c83b70-bcec-4eee-9d69-38a2fbcea88e None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:47.696530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c83b70-bcec-4eee-9d69-38a2fbcea88e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:47.696985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c83b70-bcec-4eee-9d69-38a2fbcea88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:47.721401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-f3c83b70-bcec-4eee-9d69-38a2fbcea88e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:47.722713 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f3c83b70-bcec-4eee-9d69-38a2fbcea88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:47.762578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0af7c1-c2bb-4898-850d-50154670f9cc None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:47.796751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde00e3-1760-424f-813f-7756039a1641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:47.797408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde00e3-1760-424f-813f-7756039a1641 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:47.797800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde00e3-1760-424f-813f-7756039a1641 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:47.798003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde00e3-1760-424f-813f-7756039a1641 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:47.804961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 327/634] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:47.810758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c83b70-bcec-4eee-9d69-38a2fbcea88e None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:47.870085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde00e3-1760-424f-813f-7756039a1641 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:47.871576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde00e3-1760-424f-813f-7756039a1641 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:47.952440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c83b70-bcec-4eee-9d69-38a2fbcea88e None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:47.978192 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 308/635] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:47.979042 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde00e3-1760-424f-813f-7756039a1641 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:47.997982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d5535-13a5-49ce-94ad-235f17ef9c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:47.999412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d5535-13a5-49ce-94ad-235f17ef9c32 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:47.999673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d5535-13a5-49ce-94ad-235f17ef9c32 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:47.999942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d5535-13a5-49ce-94ad-235f17ef9c32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:48.011856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d5535-13a5-49ce-94ad-235f17ef9c32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:48.013289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d5535-13a5-49ce-94ad-235f17ef9c32 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-8b0d5535-13a5-49ce-94ad-235f17ef9c32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:48.030789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:48.031860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:48.032740 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:48.032740 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:48.032740 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:48.032740 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:48.036672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 328/636] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:48.037558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d5535-13a5-49ce-94ad-235f17ef9c32 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:48.091109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde00e3-1760-424f-813f-7756039a1641 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:48.123910 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96285721-0d07-4ce8-927a-f8bf2c552caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:48.124705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96285721-0d07-4ce8-927a-f8bf2c552caf None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:48.124996 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96285721-0d07-4ce8-927a-f8bf2c552caf None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:48.125273 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96285721-0d07-4ce8-927a-f8bf2c552caf None admin] PATH_INFO: `/v3/roles/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:48.135334 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d5535-13a5-49ce-94ad-235f17ef9c32 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:48.136323 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96285721-0d07-4ce8-927a-f8bf2c552caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-96285721-0d07-4ce8-927a-f8bf2c552caf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:48.144141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:48.145066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:48.145905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:48.145905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:48.145905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:48.145905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:48.145905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:48.145905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:48.145905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:48.145905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:48.145905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:48.145905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:48.145905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:48.145905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:48.145905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 309/637] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:48.146591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96285721-0d07-4ce8-927a-f8bf2c552caf None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:48.173503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494ab0b-c98d-4efe-8ded-2eb5083d00a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:48.174828 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494ab0b-c98d-4efe-8ded-2eb5083d00a1 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:48.175105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494ab0b-c98d-4efe-8ded-2eb5083d00a1 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:48.175406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494ab0b-c98d-4efe-8ded-2eb5083d00a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:48.184797 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494ab0b-c98d-4efe-8ded-2eb5083d00a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:48.186221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494ab0b-c98d-4efe-8ded-2eb5083d00a1 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-e494ab0b-c98d-4efe-8ded-2eb5083d00a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:48.203392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:48.204270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:48.205145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:48.205145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:48.205145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:48.205145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:48.205145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 329/638] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:48.205856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164bf00-0336-48c9-a5ce-855feb12e3b2 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:48.205951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164bf00-0336-48c9-a5ce-855feb12e3b2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:48.206217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164bf00-0336-48c9-a5ce-855feb12e3b2 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:48.207603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 330/639] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:48.212003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164bf00-0336-48c9-a5ce-855feb12e3b2 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:48.256904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96285721-0d07-4ce8-927a-f8bf2c552caf None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:48.290366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560629ca-8fed-4d1c-ac30-734da45eb457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:48.291037 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560629ca-8fed-4d1c-ac30-734da45eb457 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:48.291381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560629ca-8fed-4d1c-ac30-734da45eb457 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:48.291628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560629ca-8fed-4d1c-ac30-734da45eb457 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:48.292345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560629ca-8fed-4d1c-ac30-734da45eb457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:48.292652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560629ca-8fed-4d1c-ac30-734da45eb457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:48.294003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560629ca-8fed-4d1c-ac30-734da45eb457 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:48.306747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 310/640] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:48.306747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560629ca-8fed-4d1c-ac30-734da45eb457 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:48.317450 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164bf00-0336-48c9-a5ce-855feb12e3b2 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:48.350899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15f3a2-408c-4628-8ae6-ad4794ab75ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:48.351598 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15f3a2-408c-4628-8ae6-ad4794ab75ca None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:48.351907 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15f3a2-408c-4628-8ae6-ad4794ab75ca None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:48.352136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15f3a2-408c-4628-8ae6-ad4794ab75ca None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:48.352870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15f3a2-408c-4628-8ae6-ad4794ab75ca None admin] RBAC: Adding query filter params (name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:48.353284 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15f3a2-408c-4628-8ae6-ad4794ab75ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:48.354659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15f3a2-408c-4628-8ae6-ad4794ab75ca None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:48.363593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 331/641] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:48.364709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf6e46-e8ab-4a02-ad4c-906c4a9c610b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:48.364991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf6e46-e8ab-4a02-ad4c-906c4a9c610b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:48.365265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf6e46-e8ab-4a02-ad4c-906c4a9c610b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:48.385392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c5bf6e46-e8ab-4a02-ad4c-906c4a9c610b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:48.386556 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c5bf6e46-e8ab-4a02-ad4c-906c4a9c610b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:48.412493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560629ca-8fed-4d1c-ac30-734da45eb457 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:48.445058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0524985-d470-487e-a521-e923238857f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:48.445706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0524985-d470-487e-a521-e923238857f7 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:48.445989 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0524985-d470-487e-a521-e923238857f7 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:48.446261 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0524985-d470-487e-a521-e923238857f7 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:48.446969 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0524985-d470-487e-a521-e923238857f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:48.447320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0524985-d470-487e-a521-e923238857f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:48.448657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0524985-d470-487e-a521-e923238857f7 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:48.463272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 332/642] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:48.463272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf6e46-e8ab-4a02-ad4c-906c4a9c610b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:48.469242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 311/643] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:48.470203 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0524985-d470-487e-a521-e923238857f7 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:48.564535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf6e46-e8ab-4a02-ad4c-906c4a9c610b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:48.573366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0524985-d470-487e-a521-e923238857f7 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:48.598007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cbc56-9c1c-42b2-a884-f7737de4a6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:48.598793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cbc56-9c1c-42b2-a884-f7737de4a6d8 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:48.599068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cbc56-9c1c-42b2-a884-f7737de4a6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:48.599373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cbc56-9c1c-42b2-a884-f7737de4a6d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:48.605867 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c3c3c-b00d-4e8c-a1a2-ba786072774f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:48.606209 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3cbc56-9c1c-42b2-a884-f7737de4a6d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:48.606821 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c3c3c-b00d-4e8c-a1a2-ba786072774f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:48.607170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c3c3c-b00d-4e8c-a1a2-ba786072774f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:48.607491 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c3c3c-b00d-4e8c-a1a2-ba786072774f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:48.607693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3cbc56-9c1c-42b2-a884-f7737de4a6d8 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:48.618076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c3c3c-b00d-4e8c-a1a2-ba786072774f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:48.619527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c3c3c-b00d-4e8c-a1a2-ba786072774f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-ee3cbc56-9c1c-42b2-a884-f7737de4a6d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:48.621952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:48.622992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:48.628943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:48.628943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:48.628943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:48.628943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:48.628943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:48.628943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:48.628943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:48.628943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:48.628943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 18:16:48.628943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:48.628943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 333/644] 10.222.0.25 () {56 vars in 1093 bytes} [Mon May 9 18:16:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:48.628943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cbc56-9c1c-42b2-a884-f7737de4a6d8 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-191c3c3c-b00d-4e8c-a1a2-ba786072774f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:48.636894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:48.637738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:48.639247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:48.639247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:48.639247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:48.639247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:48.639247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 312/645] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:48.643675 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c3c3c-b00d-4e8c-a1a2-ba786072774f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:48.727701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cbc56-9c1c-42b2-a884-f7737de4a6d8 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:48.742502 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c3c3c-b00d-4e8c-a1a2-ba786072774f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:48.760244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f04717-69d6-4582-8d3b-a9835e01fc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:48.761033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f04717-69d6-4582-8d3b-a9835e01fc4a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:48.761315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f04717-69d6-4582-8d3b-a9835e01fc4a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:48.761580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f04717-69d6-4582-8d3b-a9835e01fc4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:48.768601 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f04717-69d6-4582-8d3b-a9835e01fc4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:48.770011 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f04717-69d6-4582-8d3b-a9835e01fc4a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:48.775104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30697948-b02e-45f3-b365-6bf234666e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:48.775933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30697948-b02e-45f3-b365-6bf234666e58 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:48.776264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30697948-b02e-45f3-b365-6bf234666e58 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:48.776542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30697948-b02e-45f3-b365-6bf234666e58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-25f04717-69d6-4582-8d3b-a9835e01fc4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:48.783997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:48.784892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:48.785803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:48.785803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:48.785803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:48.785803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:48.785803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:48.785803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:48.785803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:48.785803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:48.786983 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30697948-b02e-45f3-b365-6bf234666e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:48.789933 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 334/646] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:48.790506 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f04717-69d6-4582-8d3b-a9835e01fc4a None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-30697948-b02e-45f3-b365-6bf234666e58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:48.795044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:48.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:48.797484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:48.797484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:48.797484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:48.797484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:48.797484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:48.797484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:48.797484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:48.797484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:48.797484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:48.797484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:48.797484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:48.797484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:48.797484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 313/647] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:48.797484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30697948-b02e-45f3-b365-6bf234666e58 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:48.889733 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f04717-69d6-4582-8d3b-a9835e01fc4a None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:48.893681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30697948-b02e-45f3-b365-6bf234666e58 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:48.922673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db595d23-0e35-45d1-941d-0837c562c4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:48.923374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db595d23-0e35-45d1-941d-0837c562c4f3 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:48.923703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db595d23-0e35-45d1-941d-0837c562c4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:48.923969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db595d23-0e35-45d1-941d-0837c562c4f3 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:48.924791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db595d23-0e35-45d1-941d-0837c562c4f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:48.925101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db595d23-0e35-45d1-941d-0837c562c4f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:48.926055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8c9bb-0991-46d1-8712-2a9085d83fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:48.926517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db595d23-0e35-45d1-941d-0837c562c4f3 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:48.926710 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8c9bb-0991-46d1-8712-2a9085d83fe8 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:48.926988 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8c9bb-0991-46d1-8712-2a9085d83fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:48.927285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8c9bb-0991-46d1-8712-2a9085d83fe8 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:48.927993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8c9bb-0991-46d1-8712-2a9085d83fe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:48.928293 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8c9bb-0991-46d1-8712-2a9085d83fe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:48.929629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8c9bb-0991-46d1-8712-2a9085d83fe8 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:48.945127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 314/648] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:48.945127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8c9bb-0991-46d1-8712-2a9085d83fe8 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:48.945388 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 335/649] 10.222.0.25 () {56 vars in 1133 bytes} [Mon May 9 18:16:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:48.945388 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db595d23-0e35-45d1-941d-0837c562c4f3 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:49.046105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db595d23-0e35-45d1-941d-0837c562c4f3 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:49.047205 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8c9bb-0991-46d1-8712-2a9085d83fe8 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:49.078342 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421ccc9-c24d-404f-ab84-4bf2dac278c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:49.079088 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421ccc9-c24d-404f-ab84-4bf2dac278c6 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:49.079402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421ccc9-c24d-404f-ab84-4bf2dac278c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:49.079685 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421ccc9-c24d-404f-ab84-4bf2dac278c6 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:49.079794 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f264de-52f7-4bc3-abf4-8485165d2a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:49.080915 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f264de-52f7-4bc3-abf4-8485165d2a22 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:49.081015 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421ccc9-c24d-404f-ab84-4bf2dac278c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:49.081188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f264de-52f7-4bc3-abf4-8485165d2a22 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:49.081291 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421ccc9-c24d-404f-ab84-4bf2dac278c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:49.081448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f264de-52f7-4bc3-abf4-8485165d2a22 None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:49.082218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f264de-52f7-4bc3-abf4-8485165d2a22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:49.082528 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f264de-52f7-4bc3-abf4-8485165d2a22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:49.082621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421ccc9-c24d-404f-ab84-4bf2dac278c6 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:49.083938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f264de-52f7-4bc3-abf4-8485165d2a22 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:49.090672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 336/650] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:49.091515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421ccc9-c24d-404f-ab84-4bf2dac278c6 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:49.102229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 315/651] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:49.102969 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f264de-52f7-4bc3-abf4-8485165d2a22 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:49.188204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421ccc9-c24d-404f-ab84-4bf2dac278c6 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:49.204718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f264de-52f7-4bc3-abf4-8485165d2a22 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:49.223536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c366e-604e-41f4-9aab-5fba62fe2a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:49.224359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c366e-604e-41f4-9aab-5fba62fe2a7f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:49.224635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c366e-604e-41f4-9aab-5fba62fe2a7f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:49.224905 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c366e-604e-41f4-9aab-5fba62fe2a7f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:49.232197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c366e-604e-41f4-9aab-5fba62fe2a7f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:49.233606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c366e-604e-41f4-9aab-5fba62fe2a7f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:49.237197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76e2c2-8aed-4b61-a15b-9b146212baa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:49.237852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76e2c2-8aed-4b61-a15b-9b146212baa9 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:49.238137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76e2c2-8aed-4b61-a15b-9b146212baa9 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:49.238414 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76e2c2-8aed-4b61-a15b-9b146212baa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-c39c366e-604e-41f4-9aab-5fba62fe2a7f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:49.247394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:49.248410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:49.249455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:49.249455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:49.249455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:49.249455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:49.249455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:49.249455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:49.249455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:49.249455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:49.249455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 18:16:49.249455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:49.249455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 337/652] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:49.250101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c366e-604e-41f4-9aab-5fba62fe2a7f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:49.309368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76e2c2-8aed-4b61-a15b-9b146212baa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:49.310885 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76e2c2-8aed-4b61-a15b-9b146212baa9 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:49.349690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c366e-604e-41f4-9aab-5fba62fe2a7f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:49.387751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdabd24-6efd-4a5f-819a-bf25e4cad9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:49.388541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdabd24-6efd-4a5f-819a-bf25e4cad9b4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:49.389117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdabd24-6efd-4a5f-819a-bf25e4cad9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:49.389117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdabd24-6efd-4a5f-819a-bf25e4cad9b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:49.399353 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdabd24-6efd-4a5f-819a-bf25e4cad9b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:49.400772 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdabd24-6efd-4a5f-819a-bf25e4cad9b4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:49.410880 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 316/653] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:49.411748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76e2c2-8aed-4b61-a15b-9b146212baa9 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-efdabd24-6efd-4a5f-819a-bf25e4cad9b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:49.418158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:49.419075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:49.420204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:49.420204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:49.420204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:49.420204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:49.420204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 338/654] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:49.420513 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdabd24-6efd-4a5f-819a-bf25e4cad9b4 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:49.513548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76e2c2-8aed-4b61-a15b-9b146212baa9 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:49.521424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdabd24-6efd-4a5f-819a-bf25e4cad9b4 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:49.550407 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c548da7-5f84-4328-b2c3-e5d921fba178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:49.551370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548da7-5f84-4328-b2c3-e5d921fba178 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:49.551724 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548da7-5f84-4328-b2c3-e5d921fba178 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:49.552072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548da7-5f84-4328-b2c3-e5d921fba178 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:49.552899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c548da7-5f84-4328-b2c3-e5d921fba178 None admin] RBAC: Adding query filter params (role.id=7b2e521b45dc4a98adad996ca0027de6, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=77cf284e81ba4042b66c0bc189f1ae2b) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:49.553209 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c548da7-5f84-4328-b2c3-e5d921fba178 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:49.554633 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c548da7-5f84-4328-b2c3-e5d921fba178 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:49.555492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e510b-cc4e-439e-9819-f0c9c4d51222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:49.556161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e510b-cc4e-439e-9819-f0c9c4d51222 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:49.556600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e510b-cc4e-439e-9819-f0c9c4d51222 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:49.556885 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e510b-cc4e-439e-9819-f0c9c4d51222 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:49.557712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e510b-cc4e-439e-9819-f0c9c4d51222 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:49.558018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e510b-cc4e-439e-9819-f0c9c4d51222 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:49.559486 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e510b-cc4e-439e-9819-f0c9c4d51222 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:49.571547 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 317/655] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:49 2022] GET /identity/v3/role_assignments?user.id=77cf284e81ba4042b66c0bc189f1ae2b&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=7b2e521b45dc4a98adad996ca0027de6 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:49.571547 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c548da7-5f84-4328-b2c3-e5d921fba178 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:49.593244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 339/656] 10.222.0.25 () {56 vars in 1137 bytes} [Mon May 9 18:16:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:49.594048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e510b-cc4e-439e-9819-f0c9c4d51222 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:49.686514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c548da7-5f84-4328-b2c3-e5d921fba178 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:49.710981 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e510b-cc4e-439e-9819-f0c9c4d51222 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:49.726418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb926641-e13a-48be-92ed-86a17a04a0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:49.727321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb926641-e13a-48be-92ed-86a17a04a0ca None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:49.727604 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb926641-e13a-48be-92ed-86a17a04a0ca None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:49.727875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb926641-e13a-48be-92ed-86a17a04a0ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:49.741325 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb926641-e13a-48be-92ed-86a17a04a0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:49.742990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb926641-e13a-48be-92ed-86a17a04a0ca None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:49.747786 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbe0bb-6bd7-4bf5-ae2e-38a5577dc175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:49.748458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbe0bb-6bd7-4bf5-ae2e-38a5577dc175 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:49.748742 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbe0bb-6bd7-4bf5-ae2e-38a5577dc175 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:49.749021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbe0bb-6bd7-4bf5-ae2e-38a5577dc175 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:49.749742 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbbe0bb-6bd7-4bf5-ae2e-38a5577dc175 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:49.750049 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbbe0bb-6bd7-4bf5-ae2e-38a5577dc175 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:49.751443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbbe0bb-6bd7-4bf5-ae2e-38a5577dc175 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-fb926641-e13a-48be-92ed-86a17a04a0ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:49.762319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:49.763252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:49.764142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:49.764142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:49.764142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:49.764142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:49.764142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 318/657] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:49.764856 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb926641-e13a-48be-92ed-86a17a04a0ca None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:49.765966 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 340/658] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:49.772637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbe0bb-6bd7-4bf5-ae2e-38a5577dc175 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:49.873979 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbe0bb-6bd7-4bf5-ae2e-38a5577dc175 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:49.879730 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb926641-e13a-48be-92ed-86a17a04a0ca None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:49.912855 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994f486-a826-4ee6-a551-37db588043bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:49.913517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994f486-a826-4ee6-a551-37db588043bb None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:49.913794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994f486-a826-4ee6-a551-37db588043bb None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:49.914069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994f486-a826-4ee6-a551-37db588043bb None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:49.914785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994f486-a826-4ee6-a551-37db588043bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:49.915083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994f486-a826-4ee6-a551-37db588043bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:49.916463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994f486-a826-4ee6-a551-37db588043bb None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:49.918537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f4d07-d716-4399-abd9-0075b0846a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:49.919205 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f4d07-d716-4399-abd9-0075b0846a4a None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:49.919520 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f4d07-d716-4399-abd9-0075b0846a4a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:49.919791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f4d07-d716-4399-abd9-0075b0846a4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:49.928971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 341/659] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:49.929787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994f486-a826-4ee6-a551-37db588043bb None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:49.997577 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f4d07-d716-4399-abd9-0075b0846a4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:49.998571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f4d07-d716-4399-abd9-0075b0846a4a None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:50.026497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994f486-a826-4ee6-a551-37db588043bb None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:50.061325 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26938c4e-4a32-437b-b7b4-28b200abb0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:50.062509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26938c4e-4a32-437b-b7b4-28b200abb0c2 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:50.063014 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26938c4e-4a32-437b-b7b4-28b200abb0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:50.063549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26938c4e-4a32-437b-b7b4-28b200abb0c2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:50.070545 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26938c4e-4a32-437b-b7b4-28b200abb0c2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:50.072050 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26938c4e-4a32-437b-b7b4-28b200abb0c2 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-26938c4e-4a32-437b-b7b4-28b200abb0c2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:50.087452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:50.088590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:50.089524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:50.089524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:50.089524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:50.089524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:50.089524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:50.089524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:50.089524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:50.089524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:50.089524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 18:16:50.089524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:50.089524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 342/660] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:50.090127 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da7d63-1fe8-4e09-bddd-3dc3e1fae71d None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:50.090615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da7d63-1fe8-4e09-bddd-3dc3e1fae71d None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:50.090615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da7d63-1fe8-4e09-bddd-3dc3e1fae71d None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:50.091958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 343/661] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:50.092556 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da7d63-1fe8-4e09-bddd-3dc3e1fae71d None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:50.107110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 319/662] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:50.107110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f4d07-d716-4399-abd9-0075b0846a4a None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:50.195016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da7d63-1fe8-4e09-bddd-3dc3e1fae71d None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:50.210975 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f4d07-d716-4399-abd9-0075b0846a4a None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:50.228343 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ec027-29e1-4875-876f-1d363c8ca358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:50.229149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec027-29e1-4875-876f-1d363c8ca358 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:50.229451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec027-29e1-4875-876f-1d363c8ca358 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:50.229726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec027-29e1-4875-876f-1d363c8ca358 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:50.236189 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ec027-29e1-4875-876f-1d363c8ca358 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:50.237616 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ec027-29e1-4875-876f-1d363c8ca358 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:50.243947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f50e0a-fea4-4bac-b3a0-077b704575b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:50.244625 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f50e0a-fea4-4bac-b3a0-077b704575b4 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:50.244910 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f50e0a-fea4-4bac-b3a0-077b704575b4 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:50.245187 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f50e0a-fea4-4bac-b3a0-077b704575b4 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:50.246022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f50e0a-fea4-4bac-b3a0-077b704575b4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:50.246328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f50e0a-fea4-4bac-b3a0-077b704575b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:50.247799 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f50e0a-fea4-4bac-b3a0-077b704575b4 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-dd7ec027-29e1-4875-876f-1d363c8ca358 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:50.250817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:50.251762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:50.259767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:50.259767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:50.259767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:50.259767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:50.259767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:50.259767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:50.259767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:50.259767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:50.259767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 344/663] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:50.259767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1dc3e-12ed-4609-a61a-1f45288d3fbd None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:50.259767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1dc3e-12ed-4609-a61a-1f45288d3fbd None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:50.260777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1dc3e-12ed-4609-a61a-1f45288d3fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:50.272895 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-5de1dc3e-12ed-4609-a61a-1f45288d3fbd None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:50.274097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-5de1dc3e-12ed-4609-a61a-1f45288d3fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:50.280211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 320/664] 10.222.0.25 () {56 vars in 1139 bytes} [Mon May 9 18:16:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:50.280211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f50e0a-fea4-4bac-b3a0-077b704575b4 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:50.355197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 345/665] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:50.355197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1dc3e-12ed-4609-a61a-1f45288d3fbd None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:50.394061 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f50e0a-fea4-4bac-b3a0-077b704575b4 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:50.427263 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e838d-1bf0-43cf-9ec0-61491549a679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:50.428066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e838d-1bf0-43cf-9ec0-61491549a679 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:50.428348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e838d-1bf0-43cf-9ec0-61491549a679 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:50.428624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e838d-1bf0-43cf-9ec0-61491549a679 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:50.438581 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e838d-1bf0-43cf-9ec0-61491549a679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:50.440612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e838d-1bf0-43cf-9ec0-61491549a679 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:50.448589 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1dc3e-12ed-4609-a61a-1f45288d3fbd None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-f24e838d-1bf0-43cf-9ec0-61491549a679 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:50.458430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:50.459353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:50.460246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:50.460246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:50.460246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:50.460246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:50.460246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 321/666] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:50.460939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e838d-1bf0-43cf-9ec0-61491549a679 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:50.488397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b7ba0-b7e1-4ba2-ab94-de469514f661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:50.489353 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b7ba0-b7e1-4ba2-ab94-de469514f661 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:50.489791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b7ba0-b7e1-4ba2-ab94-de469514f661 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:50.490065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b7ba0-b7e1-4ba2-ab94-de469514f661 None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:50.490851 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b7ba0-b7e1-4ba2-ab94-de469514f661 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:50.491158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b7ba0-b7e1-4ba2-ab94-de469514f661 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:50.492608 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b7ba0-b7e1-4ba2-ab94-de469514f661 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:50.504644 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 346/667] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:50.505966 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b7ba0-b7e1-4ba2-ab94-de469514f661 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:50.561690 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e838d-1bf0-43cf-9ec0-61491549a679 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:50.594868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6a335-8415-48a6-a0f7-20aac37e2ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:50.596078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6a335-8415-48a6-a0f7-20aac37e2ef0 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:50.596357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6a335-8415-48a6-a0f7-20aac37e2ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:50.596626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6a335-8415-48a6-a0f7-20aac37e2ef0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:50.604182 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b7ba0-b7e1-4ba2-ab94-de469514f661 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:50.636424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7055d90-ecbe-42a9-9a04-eb3365bd7b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:50.637260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7055d90-ecbe-42a9-9a04-eb3365bd7b07 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:50.637537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7055d90-ecbe-42a9-9a04-eb3365bd7b07 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:50.637860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7055d90-ecbe-42a9-9a04-eb3365bd7b07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:50.647498 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7055d90-ecbe-42a9-9a04-eb3365bd7b07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-d7055d90-ecbe-42a9-9a04-eb3365bd7b07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:50.655087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:50.656046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:50.656900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:50.656900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:50.656900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:50.656900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:50.656900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:50.656900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:50.656900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:50.656900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:50.656900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:50.656900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:50.656900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:50.656900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:50.656900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 347/668] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:50.657647 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7055d90-ecbe-42a9-9a04-eb3365bd7b07 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:50.666794 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6a335-8415-48a6-a0f7-20aac37e2ef0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:50.668269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6a335-8415-48a6-a0f7-20aac37e2ef0 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:50.752725 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7055d90-ecbe-42a9-9a04-eb3365bd7b07 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:50.767947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 322/669] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:50.767947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6a335-8415-48a6-a0f7-20aac37e2ef0 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:50.785754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e347e5-47da-4246-9b79-1fcae6c40e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:50.786434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e347e5-47da-4246-9b79-1fcae6c40e8c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:50.786715 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e347e5-47da-4246-9b79-1fcae6c40e8c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:50.786990 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e347e5-47da-4246-9b79-1fcae6c40e8c None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:50.787758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e347e5-47da-4246-9b79-1fcae6c40e8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:50.788064 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e347e5-47da-4246-9b79-1fcae6c40e8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:50.789457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e347e5-47da-4246-9b79-1fcae6c40e8c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:50.802660 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 348/670] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:50.803778 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e347e5-47da-4246-9b79-1fcae6c40e8c None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:50.867821 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6a335-8415-48a6-a0f7-20aac37e2ef0 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:50.900663 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e347e5-47da-4246-9b79-1fcae6c40e8c None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:50.901593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e29a857-42ce-4bc6-90e0-c5c5b43aa583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:50.902348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29a857-42ce-4bc6-90e0-c5c5b43aa583 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:50.902739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29a857-42ce-4bc6-90e0-c5c5b43aa583 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:50.903101 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29a857-42ce-4bc6-90e0-c5c5b43aa583 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:50.904050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e29a857-42ce-4bc6-90e0-c5c5b43aa583 None admin] RBAC: Adding query filter params (role.id=7b2e521b45dc4a98adad996ca0027de6, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=f769ee9f31354e5296645fb1e3bd84b6) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:50.904471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e29a857-42ce-4bc6-90e0-c5c5b43aa583 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:50.905975 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e29a857-42ce-4bc6-90e0-c5c5b43aa583 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:50.912541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 323/671] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:50 2022] GET /identity/v3/role_assignments?user.id=f769ee9f31354e5296645fb1e3bd84b6&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=7b2e521b45dc4a98adad996ca0027de6 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:50.913466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e29a857-42ce-4bc6-90e0-c5c5b43aa583 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:50.950166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816efbc7-fc1f-487d-9f17-1c3430cf1300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:50.951201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816efbc7-fc1f-487d-9f17-1c3430cf1300 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:50.951656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816efbc7-fc1f-487d-9f17-1c3430cf1300 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:50.952084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816efbc7-fc1f-487d-9f17-1c3430cf1300 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:50.953018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816efbc7-fc1f-487d-9f17-1c3430cf1300 None admin] RBAC: Adding query filter params (name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:50.953516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816efbc7-fc1f-487d-9f17-1c3430cf1300 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:50.955676 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816efbc7-fc1f-487d-9f17-1c3430cf1300 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:50.968962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 349/672] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:50.973472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816efbc7-fc1f-487d-9f17-1c3430cf1300 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:51.021273 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e29a857-42ce-4bc6-90e0-c5c5b43aa583 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:51.057094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19b5c7-2d30-4f67-b095-b8040dce18d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:51.058264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19b5c7-2d30-4f67-b095-b8040dce18d3 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:51.058390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19b5c7-2d30-4f67-b095-b8040dce18d3 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:51.058792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19b5c7-2d30-4f67-b095-b8040dce18d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:51.071349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19b5c7-2d30-4f67-b095-b8040dce18d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:51.072961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19b5c7-2d30-4f67-b095-b8040dce18d3 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:51.084148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816efbc7-fc1f-487d-9f17-1c3430cf1300 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-bd19b5c7-2d30-4f67-b095-b8040dce18d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:51.091593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:51.092755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:51.093876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:51.093876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:51.093876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:51.093876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:51.095479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 324/673] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:51.096963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19b5c7-2d30-4f67-b095-b8040dce18d3 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:51.123239 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a27c1e-3070-4355-8d3b-9ec2f16b181b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:51.124051 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a27c1e-3070-4355-8d3b-9ec2f16b181b None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:51.124315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a27c1e-3070-4355-8d3b-9ec2f16b181b None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:51.124585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a27c1e-3070-4355-8d3b-9ec2f16b181b None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:51.131876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a27c1e-3070-4355-8d3b-9ec2f16b181b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:51.133374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a27c1e-3070-4355-8d3b-9ec2f16b181b None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-41a27c1e-3070-4355-8d3b-9ec2f16b181b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:51.148412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:51.149339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:51.150227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:51.150227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:51.150227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:51.150227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:51.150227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:51.150227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:51.150227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:51.150227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:51.150227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 350/674] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:51.150719 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a27c1e-3070-4355-8d3b-9ec2f16b181b None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:51.216191 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19b5c7-2d30-4f67-b095-b8040dce18d3 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:51.261082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b53f1c-857a-4b63-9a16-742a095529cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:51.262127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b53f1c-857a-4b63-9a16-742a095529cf None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:51.262552 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b53f1c-857a-4b63-9a16-742a095529cf None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:51.262990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b53f1c-857a-4b63-9a16-742a095529cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:51.266809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a27c1e-3070-4355-8d3b-9ec2f16b181b None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:51.274991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b53f1c-857a-4b63-9a16-742a095529cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:51.276622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b53f1c-857a-4b63-9a16-742a095529cf None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-c3b53f1c-857a-4b63-9a16-742a095529cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:51.294015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:51.295141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:51.296250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:51.296250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:51.296250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:51.296250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:51.298090 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 325/675] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:51.299142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b53f1c-857a-4b63-9a16-742a095529cf None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:51.305996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8950a4-78b4-499d-a5c2-62f175656317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:51.306802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8950a4-78b4-499d-a5c2-62f175656317 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:51.307202 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8950a4-78b4-499d-a5c2-62f175656317 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:51.307672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8950a4-78b4-499d-a5c2-62f175656317 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:51.308504 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8950a4-78b4-499d-a5c2-62f175656317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:51.308905 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8950a4-78b4-499d-a5c2-62f175656317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:51.310366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8950a4-78b4-499d-a5c2-62f175656317 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:51.323103 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 351/676] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:51.324115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8950a4-78b4-499d-a5c2-62f175656317 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:51.400883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b53f1c-857a-4b63-9a16-742a095529cf None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:51.427474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8950a4-78b4-499d-a5c2-62f175656317 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:51.435873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f5189-159c-487e-ba51-ac32ca46e59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:51.436771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f5189-159c-487e-ba51-ac32ca46e59c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:51.437199 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f5189-159c-487e-ba51-ac32ca46e59c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:51.437695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f5189-159c-487e-ba51-ac32ca46e59c None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:51.438680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f5189-159c-487e-ba51-ac32ca46e59c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:51.439262 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f5189-159c-487e-ba51-ac32ca46e59c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:51.440872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f5189-159c-487e-ba51-ac32ca46e59c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:51.453623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 326/677] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:51.459560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f5189-159c-487e-ba51-ac32ca46e59c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:51.462475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d2597-f64b-43eb-9d8e-4312174cbd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:51.463248 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d2597-f64b-43eb-9d8e-4312174cbd33 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:51.463503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d2597-f64b-43eb-9d8e-4312174cbd33 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:51.463831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d2597-f64b-43eb-9d8e-4312174cbd33 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:51.464553 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d2597-f64b-43eb-9d8e-4312174cbd33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:51.465550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d2597-f64b-43eb-9d8e-4312174cbd33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:51.466266 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d2597-f64b-43eb-9d8e-4312174cbd33 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:51.479793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 352/678] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:51.480630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d2597-f64b-43eb-9d8e-4312174cbd33 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:51.574799 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f5189-159c-487e-ba51-ac32ca46e59c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:51.592865 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d2597-f64b-43eb-9d8e-4312174cbd33 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:51.611302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77c541-1695-4c55-8003-2ff1a99e60e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:51.612394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77c541-1695-4c55-8003-2ff1a99e60e2 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:51.612877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77c541-1695-4c55-8003-2ff1a99e60e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:51.613331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77c541-1695-4c55-8003-2ff1a99e60e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:51.622136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77c541-1695-4c55-8003-2ff1a99e60e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:51.624328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77c541-1695-4c55-8003-2ff1a99e60e2 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:51.632253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03de330-7eec-48e1-a4c9-b0040f77b5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:51.633070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03de330-7eec-48e1-a4c9-b0040f77b5bc None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:51.633442 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03de330-7eec-48e1-a4c9-b0040f77b5bc None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:51.633809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03de330-7eec-48e1-a4c9-b0040f77b5bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:51.634741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03de330-7eec-48e1-a4c9-b0040f77b5bc None admin] RBAC: Adding query filter params (role.id=7b2e521b45dc4a98adad996ca0027de6, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=4bda352a10f241ca96d5308a55238d57) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:51.635144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03de330-7eec-48e1-a4c9-b0040f77b5bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:51.636712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03de330-7eec-48e1-a4c9-b0040f77b5bc None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-3f77c541-1695-4c55-8003-2ff1a99e60e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:51.641920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:51.643542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:51.644488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:51.644488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:51.644488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:51.644488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:51.644488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:51.644488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:51.644488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:51.644488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:51.651198 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 327/679] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:51.651552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 353/680] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:51 2022] GET /identity/v3/role_assignments?user.id=4bda352a10f241ca96d5308a55238d57&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=7b2e521b45dc4a98adad996ca0027de6 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:51.652590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03de330-7eec-48e1-a4c9-b0040f77b5bc None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:51.652851 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77c541-1695-4c55-8003-2ff1a99e60e2 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:51.769590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77c541-1695-4c55-8003-2ff1a99e60e2 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:51.814617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e33423-a204-4b0c-a3fc-a8530114f224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:51.815477 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e33423-a204-4b0c-a3fc-a8530114f224 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:51.815792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e33423-a204-4b0c-a3fc-a8530114f224 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:51.816028 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e33423-a204-4b0c-a3fc-a8530114f224 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:51.817341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03de330-7eec-48e1-a4c9-b0040f77b5bc None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:51.836413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e33423-a204-4b0c-a3fc-a8530114f224 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:51.837806 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e33423-a204-4b0c-a3fc-a8530114f224 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-b6e33423-a204-4b0c-a3fc-a8530114f224 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:51.859511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:51.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:51.861511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:51.861511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:51.861511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:51.861511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:51.861511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:51.861511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:51.861511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:51.861511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:51.861511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 18:16:51.861511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:51.868475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 328/681] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:16:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:51.876212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e33423-a204-4b0c-a3fc-a8530114f224 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:51.893046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de12cb0-8e1f-445c-a243-52cc01c61c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:51.893727 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de12cb0-8e1f-445c-a243-52cc01c61c74 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:51.894004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de12cb0-8e1f-445c-a243-52cc01c61c74 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:51.894273 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de12cb0-8e1f-445c-a243-52cc01c61c74 None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:51.895052 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de12cb0-8e1f-445c-a243-52cc01c61c74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:51.895380 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de12cb0-8e1f-445c-a243-52cc01c61c74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:51.896794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de12cb0-8e1f-445c-a243-52cc01c61c74 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:51.922419 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 354/682] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:51.928316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de12cb0-8e1f-445c-a243-52cc01c61c74 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:51.992623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e33423-a204-4b0c-a3fc-a8530114f224 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:52.036023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e27e55-5a09-4565-b45a-9d8adfb704ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:52.036708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27e55-5a09-4565-b45a-9d8adfb704ac None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:52.036987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27e55-5a09-4565-b45a-9d8adfb704ac None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:52.037258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27e55-5a09-4565-b45a-9d8adfb704ac None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:52.038246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e27e55-5a09-4565-b45a-9d8adfb704ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:52.038554 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e27e55-5a09-4565-b45a-9d8adfb704ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:52.040725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e27e55-5a09-4565-b45a-9d8adfb704ac None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:52.047922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de12cb0-8e1f-445c-a243-52cc01c61c74 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:52.062870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 329/683] 10.222.0.25 () {56 vars in 1135 bytes} [Mon May 9 18:16:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:52.068328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3daf60-4466-4ccf-8c29-48b834fef6ee None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:52.069579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3daf60-4466-4ccf-8c29-48b834fef6ee None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:52.069855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3daf60-4466-4ccf-8c29-48b834fef6ee None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:52.071274 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 330/684] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:52.072242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3daf60-4466-4ccf-8c29-48b834fef6ee None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:52.096334 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa933661-52ec-49d0-9e61-f5e51ebafa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:52.097601 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa933661-52ec-49d0-9e61-f5e51ebafa62 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:52.098296 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa933661-52ec-49d0-9e61-f5e51ebafa62 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:52.098796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa933661-52ec-49d0-9e61-f5e51ebafa62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:52.100002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa933661-52ec-49d0-9e61-f5e51ebafa62 None admin] RBAC: Adding query filter params (role.id=7b2e521b45dc4a98adad996ca0027de6, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=6b2b6318776b422d9701351ba6b4ee27) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:52.100770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa933661-52ec-49d0-9e61-f5e51ebafa62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:52.102806 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa933661-52ec-49d0-9e61-f5e51ebafa62 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:52.110482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 355/685] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:51 2022] GET /identity/v3/role_assignments?user.id=6b2b6318776b422d9701351ba6b4ee27&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=7b2e521b45dc4a98adad996ca0027de6 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:52.112992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0a8f3-5250-4f56-aed8-f74ac05158de None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:52.113449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0a8f3-5250-4f56-aed8-f74ac05158de None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:52.113881 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0a8f3-5250-4f56-aed8-f74ac05158de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:52.135857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-6fe0a8f3-5250-4f56-aed8-f74ac05158de None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:52.137755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-6fe0a8f3-5250-4f56-aed8-f74ac05158de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:52.192848 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3daf60-4466-4ccf-8c29-48b834fef6ee None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:52.234388 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214573e8-bf9e-4ebf-8506-3ef8e43d845c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:52.235217 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214573e8-bf9e-4ebf-8506-3ef8e43d845c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:52.235418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214573e8-bf9e-4ebf-8506-3ef8e43d845c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:52.235698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214573e8-bf9e-4ebf-8506-3ef8e43d845c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:52.329520 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 356/686] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:52.342547 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214573e8-bf9e-4ebf-8506-3ef8e43d845c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:52.344168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214573e8-bf9e-4ebf-8506-3ef8e43d845c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:52.367308 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0a8f3-5250-4f56-aed8-f74ac05158de None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:52.464281 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 331/687] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:52.471793 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214573e8-bf9e-4ebf-8506-3ef8e43d845c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:52.492849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0a8f3-5250-4f56-aed8-f74ac05158de None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:52.528581 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cfe3a-3532-4d02-aa70-0fed836f2d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:52.529402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cfe3a-3532-4d02-aa70-0fed836f2d74 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:52.529682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cfe3a-3532-4d02-aa70-0fed836f2d74 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:52.529952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cfe3a-3532-4d02-aa70-0fed836f2d74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:52.540800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76cfe3a-3532-4d02-aa70-0fed836f2d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:52.542271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76cfe3a-3532-4d02-aa70-0fed836f2d74 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-d76cfe3a-3532-4d02-aa70-0fed836f2d74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:52.566857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:52.567869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:52.568745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:52.568745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:52.568745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:52.568745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:52.568745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 357/688] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:52.573927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cfe3a-3532-4d02-aa70-0fed836f2d74 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:52.598506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214573e8-bf9e-4ebf-8506-3ef8e43d845c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:52.645199 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc1d1f-d4ef-4a79-bf2d-863be547ee14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:52.646491 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc1d1f-d4ef-4a79-bf2d-863be547ee14 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:52.646992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc1d1f-d4ef-4a79-bf2d-863be547ee14 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:52.647499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc1d1f-d4ef-4a79-bf2d-863be547ee14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:52.658470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc1d1f-d4ef-4a79-bf2d-863be547ee14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:52.660233 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc1d1f-d4ef-4a79-bf2d-863be547ee14 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-d3fc1d1f-d4ef-4a79-bf2d-863be547ee14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:52.678302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:52.679238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:52.680230 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:52.680230 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:52.680230 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:52.680230 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:52.682465 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 332/689] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:52.687925 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc1d1f-d4ef-4a79-bf2d-863be547ee14 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:52.699662 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cfe3a-3532-4d02-aa70-0fed836f2d74 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:52.767900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b7eb0-58b3-48a9-bc01-f3f160be9852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:52.768674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b7eb0-58b3-48a9-bc01-f3f160be9852 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:52.768883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b7eb0-58b3-48a9-bc01-f3f160be9852 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:52.769162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b7eb0-58b3-48a9-bc01-f3f160be9852 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:52.769890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b7eb0-58b3-48a9-bc01-f3f160be9852 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:52.770194 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b7eb0-58b3-48a9-bc01-f3f160be9852 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:52.771583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b7eb0-58b3-48a9-bc01-f3f160be9852 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:52.787596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 358/690] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:52.790566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc1d1f-d4ef-4a79-bf2d-863be547ee14 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:52.793620 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b7eb0-58b3-48a9-bc01-f3f160be9852 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:52.829482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9ece2-a09d-40df-b61f-65651cfa885d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:52.830560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ece2-a09d-40df-b61f-65651cfa885d None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:52.831045 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ece2-a09d-40df-b61f-65651cfa885d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:52.831586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ece2-a09d-40df-b61f-65651cfa885d None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:52.832536 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9ece2-a09d-40df-b61f-65651cfa885d None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:52.833058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9ece2-a09d-40df-b61f-65651cfa885d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:52.834947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9ece2-a09d-40df-b61f-65651cfa885d None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:52.847730 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 333/691] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:52.853272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9ece2-a09d-40df-b61f-65651cfa885d None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:52.897583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b7eb0-58b3-48a9-bc01-f3f160be9852 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:52.931360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6c9aa-2627-4fd8-844b-7853ec028dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:52.932192 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6c9aa-2627-4fd8-844b-7853ec028dca None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:52.932399 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6c9aa-2627-4fd8-844b-7853ec028dca None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:52.932607 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6c9aa-2627-4fd8-844b-7853ec028dca None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:52.933310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6c9aa-2627-4fd8-844b-7853ec028dca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:52.934765 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6c9aa-2627-4fd8-844b-7853ec028dca None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:52.956716 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9ece2-a09d-40df-b61f-65651cfa885d None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:52.957523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 359/692] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:16:52 2022] POST /identity/v3/users => generated 254 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:16:53.011908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11079f9b-98ae-4980-aa49-a60f9b7d9ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:53.012776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11079f9b-98ae-4980-aa49-a60f9b7d9ca6 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:53.013053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11079f9b-98ae-4980-aa49-a60f9b7d9ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:53.013328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11079f9b-98ae-4980-aa49-a60f9b7d9ca6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:53.025262 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11079f9b-98ae-4980-aa49-a60f9b7d9ca6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:53.026767 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11079f9b-98ae-4980-aa49-a60f9b7d9ca6 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-11079f9b-98ae-4980-aa49-a60f9b7d9ca6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:53.049316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:53.050284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:53.051208 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:53.051208 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:53.051208 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:53.051208 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:53.051208 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:53.051208 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:53.051208 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:53.051208 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:53.056370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 334/693] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:53.062112 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6c9aa-2627-4fd8-844b-7853ec028dca None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:53.185739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6c9aa-2627-4fd8-844b-7853ec028dca None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:53.221659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137866be-c4b1-4bdf-b09d-bf60b1a79f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:53.222328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137866be-c4b1-4bdf-b09d-bf60b1a79f28 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:53.222604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137866be-c4b1-4bdf-b09d-bf60b1a79f28 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:53.222990 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137866be-c4b1-4bdf-b09d-bf60b1a79f28 None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:53.223855 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137866be-c4b1-4bdf-b09d-bf60b1a79f28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:53.224171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137866be-c4b1-4bdf-b09d-bf60b1a79f28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:53.225635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137866be-c4b1-4bdf-b09d-bf60b1a79f28 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:53.238389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 360/694] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:53.244601 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11079f9b-98ae-4980-aa49-a60f9b7d9ca6 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:53.389134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11079f9b-98ae-4980-aa49-a60f9b7d9ca6 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:53.426671 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbba98-ef8b-4e16-a4db-064b08ea939b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:53.427373 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbba98-ef8b-4e16-a4db-064b08ea939b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:53.427656 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbba98-ef8b-4e16-a4db-064b08ea939b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:53.427931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbba98-ef8b-4e16-a4db-064b08ea939b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:53.428765 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbba98-ef8b-4e16-a4db-064b08ea939b None admin] RBAC: Adding query filter params (role.id=7b2e521b45dc4a98adad996ca0027de6, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=ec2db3aa2b7d41f89314fec960d898b7) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:53.429073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbba98-ef8b-4e16-a4db-064b08ea939b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:53.430515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbba98-ef8b-4e16-a4db-064b08ea939b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:53.437786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 335/695] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:53 2022] GET /identity/v3/role_assignments?user.id=ec2db3aa2b7d41f89314fec960d898b7&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=7b2e521b45dc4a98adad996ca0027de6 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:53.646785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc52d79-aadf-4f84-a5b0-c95f3b4f629c None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:53.647354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc52d79-aadf-4f84-a5b0-c95f3b4f629c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:53.647354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc52d79-aadf-4f84-a5b0-c95f3b4f629c None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:53.649084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 361/696] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:53.654661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c0e13-e914-47d9-ba8c-76cf7b97c9bc None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:53.654953 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c0e13-e914-47d9-ba8c-76cf7b97c9bc None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:53.655245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c0e13-e914-47d9-ba8c-76cf7b97c9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:53.672931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-660c0e13-e914-47d9-ba8c-76cf7b97c9bc None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:53.674264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-660c0e13-e914-47d9-ba8c-76cf7b97c9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:53.749185 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 336/697] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:53.774391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc52d79-aadf-4f84-a5b0-c95f3b4f629c None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:53.892307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc52d79-aadf-4f84-a5b0-c95f3b4f629c None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:53.930361 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0507b4-c852-4854-ba98-c7837669d2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:53.931522 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0507b4-c852-4854-ba98-c7837669d2bf None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:53.932063 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0507b4-c852-4854-ba98-c7837669d2bf None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:53.932511 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0507b4-c852-4854-ba98-c7837669d2bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:53.944973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0507b4-c852-4854-ba98-c7837669d2bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-5a0507b4-c852-4854-ba98-c7837669d2bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:53.952826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:53.953774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:53.954689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:53.954689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:53.954689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:53.954689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:53.954689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:53.954689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:53.954689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:53.954689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:53.954689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:53.954689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:53.954689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:16:53.954689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:53.961542 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 362/698] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:16:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:53.965011 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c0e13-e914-47d9-ba8c-76cf7b97c9bc None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:54.060352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c0e13-e914-47d9-ba8c-76cf7b97c9bc None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:54.093790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00344e2-344a-40ad-bbcd-85d1449c1593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:54.094463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00344e2-344a-40ad-bbcd-85d1449c1593 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:54.094744 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00344e2-344a-40ad-bbcd-85d1449c1593 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:54.095018 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00344e2-344a-40ad-bbcd-85d1449c1593 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:54.095812 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00344e2-344a-40ad-bbcd-85d1449c1593 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:54.096116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00344e2-344a-40ad-bbcd-85d1449c1593 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:54.097458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00344e2-344a-40ad-bbcd-85d1449c1593 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:54.110211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 337/699] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:54.112163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0507b4-c852-4854-ba98-c7837669d2bf None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:54.244098 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0507b4-c852-4854-ba98-c7837669d2bf None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:54.288880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e88d1-4cb1-4308-bf6c-2eeb7bc6ad62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:54.289691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e88d1-4cb1-4308-bf6c-2eeb7bc6ad62 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:54.289974 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e88d1-4cb1-4308-bf6c-2eeb7bc6ad62 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:54.290248 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e88d1-4cb1-4308-bf6c-2eeb7bc6ad62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:54.306893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e88d1-4cb1-4308-bf6c-2eeb7bc6ad62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:54.308387 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e88d1-4cb1-4308-bf6c-2eeb7bc6ad62 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-035e88d1-4cb1-4308-bf6c-2eeb7bc6ad62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:54.330356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:54.331395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:54.332307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:54.332307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:54.332307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:54.332307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:54.336082 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 363/700] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:54.341370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00344e2-344a-40ad-bbcd-85d1449c1593 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:54.466272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46634219-8699-4a7f-ad4e-4035201fab1f None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:54.466568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46634219-8699-4a7f-ad4e-4035201fab1f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:54.466820 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46634219-8699-4a7f-ad4e-4035201fab1f None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:54.468199 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 364/701] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:16:54.490630 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00344e2-344a-40ad-bbcd-85d1449c1593 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:54.496681 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30344d9-7dc6-4bc6-a5a7-d0839ae3398f None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:54.496966 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30344d9-7dc6-4bc6-a5a7-d0839ae3398f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:54.497246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30344d9-7dc6-4bc6-a5a7-d0839ae3398f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:54.530807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f10b83-252f-4e1e-86a9-65ed35f3a661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:54.531498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f10b83-252f-4e1e-86a9-65ed35f3a661 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:54.531941 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f10b83-252f-4e1e-86a9-65ed35f3a661 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:54.532267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f10b83-252f-4e1e-86a9-65ed35f3a661 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:54.533531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f10b83-252f-4e1e-86a9-65ed35f3a661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:54.533832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f10b83-252f-4e1e-86a9-65ed35f3a661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:54.535207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f10b83-252f-4e1e-86a9-65ed35f3a661 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:54.541493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-e30344d9-7dc6-4bc6-a5a7-d0839ae3398f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:54.542660 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e30344d9-7dc6-4bc6-a5a7-d0839ae3398f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:54.570204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 338/702] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:54.574666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f10b83-252f-4e1e-86a9-65ed35f3a661 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:54.676759 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 365/703] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:54.716500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f10b83-252f-4e1e-86a9-65ed35f3a661 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:54.718080 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30344d9-7dc6-4bc6-a5a7-d0839ae3398f None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:54.802911 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a81532-f2e6-4559-9dbd-8e7946cca2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:54.804243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81532-f2e6-4559-9dbd-8e7946cca2fc None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:54.804666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81532-f2e6-4559-9dbd-8e7946cca2fc None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:54.805073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81532-f2e6-4559-9dbd-8e7946cca2fc None admin] PATH_INFO: `/v3/users/nova` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:54.824383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a81532-f2e6-4559-9dbd-8e7946cca2fc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:54.826646 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a81532-f2e6-4559-9dbd-8e7946cca2fc None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-c3a81532-f2e6-4559-9dbd-8e7946cca2fc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:54.869235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:54.871493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:54.873688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:54.873688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:54.873688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:54.873688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:54.873688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:54.873688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:54.873688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:54.873688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:54.873688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 18:16:54.873688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:54.876469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 339/704] 10.222.0.25 () {56 vars in 1093 bytes} [Mon May 9 18:16:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:54.878028 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a81532-f2e6-4559-9dbd-8e7946cca2fc None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:54.965877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30344d9-7dc6-4bc6-a5a7-d0839ae3398f None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:55.054525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25139d7-4262-4697-bacf-9e0c69672a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:55.055757 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25139d7-4262-4697-bacf-9e0c69672a8b None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:55.056167 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25139d7-4262-4697-bacf-9e0c69672a8b None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:55.056577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25139d7-4262-4697-bacf-9e0c69672a8b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:55.057447 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25139d7-4262-4697-bacf-9e0c69672a8b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:55.059112 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25139d7-4262-4697-bacf-9e0c69672a8b None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-e25139d7-4262-4697-bacf-9e0c69672a8b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:55.088702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:55.089731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:55.090572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 366/705] 10.222.0.25 () {56 vars in 1129 bytes} [Mon May 9 18:16:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:55.092019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1755ac-0154-428f-9170-ae5d8330b90e None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:55.092019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1755ac-0154-428f-9170-ae5d8330b90e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:55.092349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1755ac-0154-428f-9170-ae5d8330b90e None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:55.096410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 367/706] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:55.100903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88885cae-984f-48ad-b486-f9b2410a1a2f None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:55.101196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88885cae-984f-48ad-b486-f9b2410a1a2f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:55.101751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88885cae-984f-48ad-b486-f9b2410a1a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:55.106329 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a81532-f2e6-4559-9dbd-8e7946cca2fc None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:55.120985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-88885cae-984f-48ad-b486-f9b2410a1a2f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:55.122664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-88885cae-984f-48ad-b486-f9b2410a1a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:55.145584 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ad378-afa0-40ca-a6cb-78c17a462c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:55.146253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ad378-afa0-40ca-a6cb-78c17a462c12 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:55.146529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ad378-afa0-40ca-a6cb-78c17a462c12 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:55.147265 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ad378-afa0-40ca-a6cb-78c17a462c12 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:55.148379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ad378-afa0-40ca-a6cb-78c17a462c12 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:55.148687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ad378-afa0-40ca-a6cb-78c17a462c12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:55.150572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ad378-afa0-40ca-a6cb-78c17a462c12 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:55.171595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 340/707] 10.222.0.25 () {56 vars in 1133 bytes} [Mon May 9 18:16:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:55.172798 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ad378-afa0-40ca-a6cb-78c17a462c12 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:55.199269 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 368/708] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:55.200315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88885cae-984f-48ad-b486-f9b2410a1a2f None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:55.297497 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ad378-afa0-40ca-a6cb-78c17a462c12 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:55.328497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88885cae-984f-48ad-b486-f9b2410a1a2f None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:55.332314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee31b6d-61d9-4a93-a088-53b32731cdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:55.332989 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee31b6d-61d9-4a93-a088-53b32731cdc9 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:55.333267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee31b6d-61d9-4a93-a088-53b32731cdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:55.333541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee31b6d-61d9-4a93-a088-53b32731cdc9 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:55.334303 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee31b6d-61d9-4a93-a088-53b32731cdc9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:55.334609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee31b6d-61d9-4a93-a088-53b32731cdc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:55.335992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee31b6d-61d9-4a93-a088-53b32731cdc9 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:55.340791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 341/709] 10.222.0.25 () {56 vars in 1124 bytes} [Mon May 9 18:16:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:55.341586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee31b6d-61d9-4a93-a088-53b32731cdc9 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:55.368035 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f2816-10eb-42ff-a26a-fad0d6d74f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:55.369107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f2816-10eb-42ff-a26a-fad0d6d74f7e None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:55.369668 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f2816-10eb-42ff-a26a-fad0d6d74f7e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:55.369948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f2816-10eb-42ff-a26a-fad0d6d74f7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:55.447673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee31b6d-61d9-4a93-a088-53b32731cdc9 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:55.454813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f2816-10eb-42ff-a26a-fad0d6d74f7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:55.456766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f2816-10eb-42ff-a26a-fad0d6d74f7e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:55.489261 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab20753-bf0a-4662-93bb-e2a0798fb5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:55.490194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab20753-bf0a-4662-93bb-e2a0798fb5e8 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:55.490474 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab20753-bf0a-4662-93bb-e2a0798fb5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:55.490749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab20753-bf0a-4662-93bb-e2a0798fb5e8 None admin] PATH_INFO: `/v3/services/network` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:55.491592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab20753-bf0a-4662-93bb-e2a0798fb5e8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:55.493746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab20753-bf0a-4662-93bb-e2a0798fb5e8 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-4ab20753-bf0a-4662-93bb-e2a0798fb5e8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:55.511694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:55.513992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:55.515884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 342/710] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:55 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:55.517717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab20753-bf0a-4662-93bb-e2a0798fb5e8 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:55.562980 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 369/711] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 18:16:55.563612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f2816-10eb-42ff-a26a-fad0d6d74f7e None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:55.649946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab20753-bf0a-4662-93bb-e2a0798fb5e8 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:55.663820 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f2816-10eb-42ff-a26a-fad0d6d74f7e None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:55.692934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76333be-a824-4829-abb9-0e1e08b4d206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:55.694255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76333be-a824-4829-abb9-0e1e08b4d206 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:55.694626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76333be-a824-4829-abb9-0e1e08b4d206 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:55.694976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76333be-a824-4829-abb9-0e1e08b4d206 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:55.696076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76333be-a824-4829-abb9-0e1e08b4d206 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:55.696478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76333be-a824-4829-abb9-0e1e08b4d206 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:55.697970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76333be-a824-4829-abb9-0e1e08b4d206 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:55.698207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a288e1-7c61-4b76-b875-6e80a625dcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:55.698873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a288e1-7c61-4b76-b875-6e80a625dcbd None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:55.699157 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a288e1-7c61-4b76-b875-6e80a625dcbd None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:55.699469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a288e1-7c61-4b76-b875-6e80a625dcbd None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:55.700169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a288e1-7c61-4b76-b875-6e80a625dcbd None admin] RBAC: Adding query filter params (name=network) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:55.700468 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a288e1-7c61-4b76-b875-6e80a625dcbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:55.701867 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a288e1-7c61-4b76-b875-6e80a625dcbd None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:55.704452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 343/712] 10.222.0.25 () {56 vars in 1124 bytes} [Mon May 9 18:16:55 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:55.705418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76333be-a824-4829-abb9-0e1e08b4d206 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:55.714096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 370/713] 10.222.0.25 () {56 vars in 1112 bytes} [Mon May 9 18:16:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:55.719470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a288e1-7c61-4b76-b875-6e80a625dcbd None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:55.825958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76333be-a824-4829-abb9-0e1e08b4d206 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:55.870266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05466168-656d-4d9e-a9b8-a49d42d5b29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:55.871089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05466168-656d-4d9e-a9b8-a49d42d5b29a None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:55.871403 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05466168-656d-4d9e-a9b8-a49d42d5b29a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:55.871685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05466168-656d-4d9e-a9b8-a49d42d5b29a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:55.904387 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05466168-656d-4d9e-a9b8-a49d42d5b29a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:55.905491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a288e1-7c61-4b76-b875-6e80a625dcbd None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:55.905850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05466168-656d-4d9e-a9b8-a49d42d5b29a None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-05466168-656d-4d9e-a9b8-a49d42d5b29a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:55.947499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:55.949463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:55.950626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:55.950626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:55.950626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:55.950626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:55.950626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 344/714] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:55.952790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d56ae-6711-49f5-b274-0ba4893b29b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:55.954206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05466168-656d-4d9e-a9b8-a49d42d5b29a None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:55.955074 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d56ae-6711-49f5-b274-0ba4893b29b7 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:55.955383 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d56ae-6711-49f5-b274-0ba4893b29b7 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:55.955649 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d56ae-6711-49f5-b274-0ba4893b29b7 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:55.961252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d56ae-6711-49f5-b274-0ba4893b29b7 None admin] RBAC: Adding query filter params (type=network) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:55.961252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d56ae-6711-49f5-b274-0ba4893b29b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:55.961252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d56ae-6711-49f5-b274-0ba4893b29b7 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:55.976453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 371/715] 10.222.0.25 () {56 vars in 1112 bytes} [Mon May 9 18:16:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:55.985224 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4f0ef-2f04-4a06-b44a-78dac2096122 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:55.986700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4f0ef-2f04-4a06-b44a-78dac2096122 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:55.986992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4f0ef-2f04-4a06-b44a-78dac2096122 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:55.989037 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 372/716] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:16:55.995955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9334123-7981-44c4-bb06-a57c8ec6fcc0 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:55.996246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9334123-7981-44c4-bb06-a57c8ec6fcc0 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:55.996511 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9334123-7981-44c4-bb06-a57c8ec6fcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:56.022742 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c9334123-7981-44c4-bb06-a57c8ec6fcc0 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:56.024319 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c9334123-7981-44c4-bb06-a57c8ec6fcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:56.088217 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05466168-656d-4d9e-a9b8-a49d42d5b29a None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:56.125342 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 373/717] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:56.136433 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430262f-2e33-498a-8c10-12b2fd9daeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:56.137186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430262f-2e33-498a-8c10-12b2fd9daeed None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:56.137472 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430262f-2e33-498a-8c10-12b2fd9daeed None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:56.137749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430262f-2e33-498a-8c10-12b2fd9daeed None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:56.138537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430262f-2e33-498a-8c10-12b2fd9daeed None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:56.138872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430262f-2e33-498a-8c10-12b2fd9daeed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:56.140434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430262f-2e33-498a-8c10-12b2fd9daeed None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:56.156653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 345/718] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:56.162270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9334123-7981-44c4-bb06-a57c8ec6fcc0 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:56.193540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430262f-2e33-498a-8c10-12b2fd9daeed None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:56.268760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9334123-7981-44c4-bb06-a57c8ec6fcc0 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:56.301297 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430262f-2e33-498a-8c10-12b2fd9daeed None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:56.305831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95b7e0-6033-4b6a-9604-cf88154f8e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:56.306632 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95b7e0-6033-4b6a-9604-cf88154f8e25 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:56.306908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95b7e0-6033-4b6a-9604-cf88154f8e25 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:56.307196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95b7e0-6033-4b6a-9604-cf88154f8e25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:56.314437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95b7e0-6033-4b6a-9604-cf88154f8e25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:56.315991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95b7e0-6033-4b6a-9604-cf88154f8e25 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-1f95b7e0-6033-4b6a-9604-cf88154f8e25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:56.332179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:56.334264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:56.336433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:56.336433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:56.336433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:56.336433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:56.336433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:56.336433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:56.336433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:56.336433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:56.336433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 374/719] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:56.337573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51622286-79c2-4247-bfc6-4c4113b05f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:56.337741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51622286-79c2-4247-bfc6-4c4113b05f98 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:56.337883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51622286-79c2-4247-bfc6-4c4113b05f98 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:56.338345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51622286-79c2-4247-bfc6-4c4113b05f98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:56.347265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95b7e0-6033-4b6a-9604-cf88154f8e25 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:56.349697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51622286-79c2-4247-bfc6-4c4113b05f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-51622286-79c2-4247-bfc6-4c4113b05f98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:56.357609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:56.359814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:56.361819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:56.361819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:56.361819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:56.361819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:56.361819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:56.361819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:56.361819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:56.361819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:56.361819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:56.361819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:56.361819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:16:56.361819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:56.361819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 346/720] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:16:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:56.371216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51622286-79c2-4247-bfc6-4c4113b05f98 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:56.442647 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95b7e0-6033-4b6a-9604-cf88154f8e25 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:56.479609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8001fb6-2477-4400-886b-32f057821013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:56.480269 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8001fb6-2477-4400-886b-32f057821013 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:56.480551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8001fb6-2477-4400-886b-32f057821013 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:56.480934 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8001fb6-2477-4400-886b-32f057821013 None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:56.481779 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8001fb6-2477-4400-886b-32f057821013 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:56.482139 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8001fb6-2477-4400-886b-32f057821013 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:56.483672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8001fb6-2477-4400-886b-32f057821013 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:56.496444 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 375/721] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:56.503109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8001fb6-2477-4400-886b-32f057821013 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:56.555231 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51622286-79c2-4247-bfc6-4c4113b05f98 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:56.591355 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2329b48-47f6-4464-ab83-37e01f9c57ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:56.591956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2329b48-47f6-4464-ab83-37e01f9c57ee None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:56.592240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2329b48-47f6-4464-ab83-37e01f9c57ee None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:56.592518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2329b48-47f6-4464-ab83-37e01f9c57ee None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:56.593777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2329b48-47f6-4464-ab83-37e01f9c57ee None admin] RBAC: Adding query filter params (name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:56.594281 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2329b48-47f6-4464-ab83-37e01f9c57ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:56.596287 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2329b48-47f6-4464-ab83-37e01f9c57ee None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:56.604915 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 347/722] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:16:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:56.606420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ab845-d66a-4518-993a-04d86202dbff None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:56.606549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ab845-d66a-4518-993a-04d86202dbff None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:56.606822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ab845-d66a-4518-993a-04d86202dbff None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:56.609728 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8001fb6-2477-4400-886b-32f057821013 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:56.612915 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 348/723] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:56.613051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ab845-d66a-4518-993a-04d86202dbff None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:56.646148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b755e-ca55-4e18-b75b-adadee1fbe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:56.647149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b755e-ca55-4e18-b75b-adadee1fbe98 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:56.647666 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b755e-ca55-4e18-b75b-adadee1fbe98 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:56.648136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b755e-ca55-4e18-b75b-adadee1fbe98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:56.649153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b755e-ca55-4e18-b75b-adadee1fbe98 None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=f769ee9f31354e5296645fb1e3bd84b6) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:56.649757 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b755e-ca55-4e18-b75b-adadee1fbe98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:56.651472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b755e-ca55-4e18-b75b-adadee1fbe98 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:56.657686 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 376/724] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:56 2022] GET /identity/v3/role_assignments?user.id=f769ee9f31354e5296645fb1e3bd84b6&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:56.659369 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b474a-c057-4cfa-b5cc-d8dd019fe61a None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:56.659888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b474a-c057-4cfa-b5cc-d8dd019fe61a None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:56.660411 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b474a-c057-4cfa-b5cc-d8dd019fe61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:56.677913 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4f7b474a-c057-4cfa-b5cc-d8dd019fe61a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:56.679577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4f7b474a-c057-4cfa-b5cc-d8dd019fe61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:56.736414 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ab845-d66a-4518-993a-04d86202dbff None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:56.797019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0ea9a-8e5f-4b8d-9b80-b452b58e136d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:56.797973 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0ea9a-8e5f-4b8d-9b80-b452b58e136d None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:56.798286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0ea9a-8e5f-4b8d-9b80-b452b58e136d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:56.798572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0ea9a-8e5f-4b8d-9b80-b452b58e136d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:56.817143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0ea9a-8e5f-4b8d-9b80-b452b58e136d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:56.819920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0ea9a-8e5f-4b8d-9b80-b452b58e136d None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:56.844979 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 377/725] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-15b0ea9a-8e5f-4b8d-9b80-b452b58e136d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:56.852255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:56.853174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:56.854093 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:56.854093 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:56.854093 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:56.854093 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:56.854093 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 349/726] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:56.859097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b474a-c057-4cfa-b5cc-d8dd019fe61a None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:56.877606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0ea9a-8e5f-4b8d-9b80-b452b58e136d None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:56.970187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b474a-c057-4cfa-b5cc-d8dd019fe61a None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:57.022342 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0ea9a-8e5f-4b8d-9b80-b452b58e136d None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:57.027418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a1b72-1f49-43fa-89a5-bd03b49df9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:57.028087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a1b72-1f49-43fa-89a5-bd03b49df9fc None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:57.028365 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a1b72-1f49-43fa-89a5-bd03b49df9fc None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:57.028639 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a1b72-1f49-43fa-89a5-bd03b49df9fc None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:57.029409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a1b72-1f49-43fa-89a5-bd03b49df9fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:57.029708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a1b72-1f49-43fa-89a5-bd03b49df9fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:57.031114 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a1b72-1f49-43fa-89a5-bd03b49df9fc None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:57.049612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 378/727] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:57.054544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a1b72-1f49-43fa-89a5-bd03b49df9fc None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:57.070347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb2eb5-1f14-4059-855e-13a16ec19bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:57.071509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb2eb5-1f14-4059-855e-13a16ec19bca None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:57.071910 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb2eb5-1f14-4059-855e-13a16ec19bca None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:57.072297 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb2eb5-1f14-4059-855e-13a16ec19bca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:57.086293 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb2eb5-1f14-4059-855e-13a16ec19bca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-73bb2eb5-1f14-4059-855e-13a16ec19bca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:16:57.098776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:57.099771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:57.100629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:57.100629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:57.100629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:57.100629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:57.100629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:16:57.100629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:16:57.100629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:16:57.100629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:16:57.100629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:16:57.100629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:16:57.100629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:16:57.100629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:57.106997 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 350/728] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:16:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:57.108626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb2eb5-1f14-4059-855e-13a16ec19bca None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:57.162838 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a1b72-1f49-43fa-89a5-bd03b49df9fc None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:57.198032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd472d31-24aa-43f8-af50-f7c8c0e0594f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:57.198842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd472d31-24aa-43f8-af50-f7c8c0e0594f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:57.199120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd472d31-24aa-43f8-af50-f7c8c0e0594f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:57.199420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd472d31-24aa-43f8-af50-f7c8c0e0594f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:57.207568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd472d31-24aa-43f8-af50-f7c8c0e0594f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:57.208996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd472d31-24aa-43f8-af50-f7c8c0e0594f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-fd472d31-24aa-43f8-af50-f7c8c0e0594f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:57.224316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:57.226490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:57.228853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:57.228853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:57.228853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:57.228853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:57.228853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:57.228853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:57.228853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:57.228853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:57.228853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 18:16:57.228853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:57.228853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 379/729] 10.222.0.25 () {56 vars in 1117 bytes} [Mon May 9 18:16:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:16:57.235198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd472d31-24aa-43f8-af50-f7c8c0e0594f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:57.261662 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb2eb5-1f14-4059-855e-13a16ec19bca None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:57.349302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20455971-7d81-4ff9-a5eb-e57df1b6e4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:57.349907 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20455971-7d81-4ff9-a5eb-e57df1b6e4d6 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:57.350522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20455971-7d81-4ff9-a5eb-e57df1b6e4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:57.350522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20455971-7d81-4ff9-a5eb-e57df1b6e4d6 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:57.351196 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20455971-7d81-4ff9-a5eb-e57df1b6e4d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:57.351512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20455971-7d81-4ff9-a5eb-e57df1b6e4d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:57.352870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20455971-7d81-4ff9-a5eb-e57df1b6e4d6 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:57.369709 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 351/730] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:16:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:57.372171 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20455971-7d81-4ff9-a5eb-e57df1b6e4d6 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:57.453626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd472d31-24aa-43f8-af50-f7c8c0e0594f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:57.502984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20455971-7d81-4ff9-a5eb-e57df1b6e4d6 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:57.520833 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bedfed-8389-4d42-91c4-90f8b1c33b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:57.523849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bedfed-8389-4d42-91c4-90f8b1c33b37 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:57.523849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bedfed-8389-4d42-91c4-90f8b1c33b37 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:57.523849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bedfed-8389-4d42-91c4-90f8b1c33b37 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:57.523849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bedfed-8389-4d42-91c4-90f8b1c33b37 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:57.523849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bedfed-8389-4d42-91c4-90f8b1c33b37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:57.524488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bedfed-8389-4d42-91c4-90f8b1c33b37 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:57.552610 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca36b7e-af51-4c72-897e-303cf25d924c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:57.553758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca36b7e-af51-4c72-897e-303cf25d924c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:57.554222 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca36b7e-af51-4c72-897e-303cf25d924c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:57.554677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca36b7e-af51-4c72-897e-303cf25d924c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:57.566334 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca36b7e-af51-4c72-897e-303cf25d924c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:57.568096 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca36b7e-af51-4c72-897e-303cf25d924c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:57.569746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 380/731] 10.222.0.25 () {56 vars in 1149 bytes} [Mon May 9 18:16:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:57.579403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bedfed-8389-4d42-91c4-90f8b1c33b37 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-6ca36b7e-af51-4c72-897e-303cf25d924c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:57.588107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:57.589078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:57.590003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:57.590003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:57.590003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:57.590003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:57.597023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 352/732] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:57.598523 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca36b7e-af51-4c72-897e-303cf25d924c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:57.706953 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca36b7e-af51-4c72-897e-303cf25d924c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:57.748289 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30ac52-c845-4c56-b06f-9dd1783eb6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:57.749273 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30ac52-c845-4c56-b06f-9dd1783eb6e5 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:57.749724 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30ac52-c845-4c56-b06f-9dd1783eb6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:57.750147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30ac52-c845-4c56-b06f-9dd1783eb6e5 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:57.751019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30ac52-c845-4c56-b06f-9dd1783eb6e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:57.751527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30ac52-c845-4c56-b06f-9dd1783eb6e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:57.753314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30ac52-c845-4c56-b06f-9dd1783eb6e5 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:57.771302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 353/733] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:57.772868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30ac52-c845-4c56-b06f-9dd1783eb6e5 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:57.873157 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bedfed-8389-4d42-91c4-90f8b1c33b37 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:57.877706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30ac52-c845-4c56-b06f-9dd1783eb6e5 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:57.915379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821357c-ee4d-49c4-910f-5e41221db125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:57.916873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821357c-ee4d-49c4-910f-5e41221db125 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:57.917451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821357c-ee4d-49c4-910f-5e41221db125 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:57.917876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821357c-ee4d-49c4-910f-5e41221db125 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:57.925651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821357c-ee4d-49c4-910f-5e41221db125 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:57.927299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821357c-ee4d-49c4-910f-5e41221db125 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-f821357c-ee4d-49c4-910f-5e41221db125 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:16:57.943713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:57.944869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:57.945982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:57.945982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:57.945982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:16:57.945982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:16:57.945982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:16:57.945982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:16:57.945982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:16:57.945982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:16:57.945982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 18:16:57.945982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:57.953013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 354/734] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:16:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:57.956162 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821357c-ee4d-49c4-910f-5e41221db125 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:57.983210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bf593-10eb-4437-b935-3ccd3fc938ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:57.983210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bf593-10eb-4437-b935-3ccd3fc938ab None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:57.983210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bf593-10eb-4437-b935-3ccd3fc938ab None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:57.983210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bf593-10eb-4437-b935-3ccd3fc938ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:58.051438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bf593-10eb-4437-b935-3ccd3fc938ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:58.053354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bf593-10eb-4437-b935-3ccd3fc938ab None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:58.135847 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821357c-ee4d-49c4-910f-5e41221db125 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:58.154477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 381/735] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 18:16:58.169246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bf593-10eb-4437-b935-3ccd3fc938ab None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:58.188461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5004a04-8b89-4e31-bef1-4aeb059002d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:58.189684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5004a04-8b89-4e31-bef1-4aeb059002d9 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:58.189967 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5004a04-8b89-4e31-bef1-4aeb059002d9 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:58.190240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5004a04-8b89-4e31-bef1-4aeb059002d9 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:58.191085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5004a04-8b89-4e31-bef1-4aeb059002d9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:58.191423 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5004a04-8b89-4e31-bef1-4aeb059002d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:58.192843 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5004a04-8b89-4e31-bef1-4aeb059002d9 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:58.227847 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 355/736] 10.222.0.25 () {56 vars in 1135 bytes} [Mon May 9 18:16:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:58.241693 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5004a04-8b89-4e31-bef1-4aeb059002d9 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:58.278160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bf593-10eb-4437-b935-3ccd3fc938ab None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:58.312330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7ae1c-4c75-4449-b992-d86a3d634d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:58.313178 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7ae1c-4c75-4449-b992-d86a3d634d38 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:58.313466 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7ae1c-4c75-4449-b992-d86a3d634d38 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:58.313741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7ae1c-4c75-4449-b992-d86a3d634d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:58.324758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7ae1c-4c75-4449-b992-d86a3d634d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:58.326210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7ae1c-4c75-4449-b992-d86a3d634d38 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-73f7ae1c-4c75-4449-b992-d86a3d634d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:58.344816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:58.345785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:58.346672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:58.346672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:58.346672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:58.346672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:58.346891 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5004a04-8b89-4e31-bef1-4aeb059002d9 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:58.346965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 382/737] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:58.356188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7ae1c-4c75-4449-b992-d86a3d634d38 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:58.390901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888d514-ab13-4843-a035-82f72444f614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:58.391591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888d514-ab13-4843-a035-82f72444f614 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:58.391878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888d514-ab13-4843-a035-82f72444f614 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:58.392459 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888d514-ab13-4843-a035-82f72444f614 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:58.505853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888d514-ab13-4843-a035-82f72444f614 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:58.507333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888d514-ab13-4843-a035-82f72444f614 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:58.585595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7ae1c-4c75-4449-b992-d86a3d634d38 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:58.608783 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 356/738] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 18:16:58.614711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f21914-f838-4e9c-b9d7-6669c719caab None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:58.616302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f21914-f838-4e9c-b9d7-6669c719caab None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:58.616959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f21914-f838-4e9c-b9d7-6669c719caab None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:58.620277 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 357/739] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:16:58.621371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f21914-f838-4e9c-b9d7-6669c719caab None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:58.642644 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b55af9-7fde-48b4-a43f-70b33ea7ca9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:58.643349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b55af9-7fde-48b4-a43f-70b33ea7ca9c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:58.643631 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b55af9-7fde-48b4-a43f-70b33ea7ca9c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:58.643901 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b55af9-7fde-48b4-a43f-70b33ea7ca9c None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:58.644608 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b55af9-7fde-48b4-a43f-70b33ea7ca9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:58.644910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b55af9-7fde-48b4-a43f-70b33ea7ca9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:58.646259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b55af9-7fde-48b4-a43f-70b33ea7ca9c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:58.659879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 383/740] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:58.661386 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69248e76-782e-46a9-a57c-a27786280b1e None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:58.661669 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69248e76-782e-46a9-a57c-a27786280b1e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:58.661942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69248e76-782e-46a9-a57c-a27786280b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:58.681790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-69248e76-782e-46a9-a57c-a27786280b1e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:58.683028 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-69248e76-782e-46a9-a57c-a27786280b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:58.739553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f21914-f838-4e9c-b9d7-6669c719caab None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:58.750909 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 384/741] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:58.752736 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69248e76-782e-46a9-a57c-a27786280b1e None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:58.778952 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a99cdd-6721-4d40-950d-766291dafe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:58.780175 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a99cdd-6721-4d40-950d-766291dafe1f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:58.780660 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a99cdd-6721-4d40-950d-766291dafe1f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:58.781143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a99cdd-6721-4d40-950d-766291dafe1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:58.805729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a99cdd-6721-4d40-950d-766291dafe1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:58.808168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a99cdd-6721-4d40-950d-766291dafe1f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-75a99cdd-6721-4d40-950d-766291dafe1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:16:58.832515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:58.833503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:58.834418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:16:58.834418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:16:58.834418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:16:58.834418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:58.834418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 358/742] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:58.835136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a99cdd-6721-4d40-950d-766291dafe1f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:58.862653 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69248e76-782e-46a9-a57c-a27786280b1e None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:58.897160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7beb1a8-423b-45a1-a95c-e0ef362dc8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:58.898639 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beb1a8-423b-45a1-a95c-e0ef362dc8d0 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:58.898910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beb1a8-423b-45a1-a95c-e0ef362dc8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:58.899202 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beb1a8-423b-45a1-a95c-e0ef362dc8d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:58.905795 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7beb1a8-423b-45a1-a95c-e0ef362dc8d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:58.907235 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7beb1a8-423b-45a1-a95c-e0ef362dc8d0 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-b7beb1a8-423b-45a1-a95c-e0ef362dc8d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:58.921231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:58.922188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:58.923083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:58.923083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:58.923083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:58.923083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:58.923083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:58.923083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:58.923083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:58.923083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:16:58.927348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 385/743] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:58.928859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9df9bc-7680-4a52-9247-6ff74e9c133d None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:58.929148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9df9bc-7680-4a52-9247-6ff74e9c133d None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:58.929424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9df9bc-7680-4a52-9247-6ff74e9c133d None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:58.930824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 386/744] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:58.931405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9df9bc-7680-4a52-9247-6ff74e9c133d None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:58.942582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a99cdd-6721-4d40-950d-766291dafe1f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:58.981292 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655e0aa-cb47-4b3b-a394-bef8f6406152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:58.981961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655e0aa-cb47-4b3b-a394-bef8f6406152 None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:58.982245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655e0aa-cb47-4b3b-a394-bef8f6406152 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:58.982519 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655e0aa-cb47-4b3b-a394-bef8f6406152 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:58.983256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655e0aa-cb47-4b3b-a394-bef8f6406152 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:58.984620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655e0aa-cb47-4b3b-a394-bef8f6406152 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:58.999829 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 359/745] 10.222.0.25 () {60 vars in 1141 bytes} [Mon May 9 18:16:58 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:16:59.000645 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655e0aa-cb47-4b3b-a394-bef8f6406152 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:59.071734 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9df9bc-7680-4a52-9247-6ff74e9c133d None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:59.111771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c6d4a-6459-4536-9a06-3e5b3fd7dcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:59.112491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c6d4a-6459-4536-9a06-3e5b3fd7dcf9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:59.112769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c6d4a-6459-4536-9a06-3e5b3fd7dcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:59.113046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c6d4a-6459-4536-9a06-3e5b3fd7dcf9 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:59.113832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c6d4a-6459-4536-9a06-3e5b3fd7dcf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:59.114136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c6d4a-6459-4536-9a06-3e5b3fd7dcf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:59.115677 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c6d4a-6459-4536-9a06-3e5b3fd7dcf9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:59.129222 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 387/746] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:59.130285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655e0aa-cb47-4b3b-a394-bef8f6406152 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:59.130695 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191be5f-5855-4dd4-95e4-15dd918383f3 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:59.130988 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191be5f-5855-4dd4-95e4-15dd918383f3 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:59.131347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191be5f-5855-4dd4-95e4-15dd918383f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:59.149963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-d191be5f-5855-4dd4-95e4-15dd918383f3 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:59.151318 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-d191be5f-5855-4dd4-95e4-15dd918383f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:59.174927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd375bf4-d7a6-4446-936b-096efd94b59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:59.176286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd375bf4-d7a6-4446-936b-096efd94b59b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:59.176754 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd375bf4-d7a6-4446-936b-096efd94b59b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:59.177346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd375bf4-d7a6-4446-936b-096efd94b59b None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:59.179001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd375bf4-d7a6-4446-936b-096efd94b59b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:59.179590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd375bf4-d7a6-4446-936b-096efd94b59b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:59.182062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd375bf4-d7a6-4446-936b-096efd94b59b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:59.203216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 360/747] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:59.203216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd375bf4-d7a6-4446-936b-096efd94b59b None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:59.232080 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 388/748] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 18:16:59.232289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191be5f-5855-4dd4-95e4-15dd918383f3 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:59.338687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191be5f-5855-4dd4-95e4-15dd918383f3 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:59.342167 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd375bf4-d7a6-4446-936b-096efd94b59b None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:59.372459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fdebe8-cd6f-4625-9ee7-21bf5bda0257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:59.373115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdebe8-cd6f-4625-9ee7-21bf5bda0257 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:59.373398 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdebe8-cd6f-4625-9ee7-21bf5bda0257 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:59.373670 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdebe8-cd6f-4625-9ee7-21bf5bda0257 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:59.374569 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fdebe8-cd6f-4625-9ee7-21bf5bda0257 None admin] RBAC: Adding query filter params (role.id=7b2e521b45dc4a98adad996ca0027de6, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=154f86b7575e47efb2a5e1c35943e037) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:59.374870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fdebe8-cd6f-4625-9ee7-21bf5bda0257 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:59.376354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fdebe8-cd6f-4625-9ee7-21bf5bda0257 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:59.382634 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 389/749] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:59 2022] GET /identity/v3/role_assignments?user.id=154f86b7575e47efb2a5e1c35943e037&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=7b2e521b45dc4a98adad996ca0027de6 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:59.383671 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e298d-9290-4a0a-add1-99691dce870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:59.384617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e298d-9290-4a0a-add1-99691dce870c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:59.384902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e298d-9290-4a0a-add1-99691dce870c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:59.385183 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e298d-9290-4a0a-add1-99691dce870c None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:59.387804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fdebe8-cd6f-4625-9ee7-21bf5bda0257 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:59.394965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e298d-9290-4a0a-add1-99691dce870c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:59.397515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e298d-9290-4a0a-add1-99691dce870c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-9b7e298d-9290-4a0a-add1-99691dce870c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:16:59.414123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:16:59.415135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:16:59.416179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:16:59.416179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:16:59.416179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:16:59.416179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:16:59.416179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:16:59.416179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:16:59.416179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:16:59.416179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:16:59.420511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 361/750] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:16:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:16:59.421963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e298d-9290-4a0a-add1-99691dce870c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:59.533879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e298d-9290-4a0a-add1-99691dce870c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:59.548909 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fdebe8-cd6f-4625-9ee7-21bf5bda0257 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:59.572502 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3baf10-7d39-4347-9ad7-ddb14b0d606c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:59.573186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3baf10-7d39-4347-9ad7-ddb14b0d606c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:59.573473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3baf10-7d39-4347-9ad7-ddb14b0d606c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:59.573755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3baf10-7d39-4347-9ad7-ddb14b0d606c None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:59.574553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3baf10-7d39-4347-9ad7-ddb14b0d606c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:59.574865 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3baf10-7d39-4347-9ad7-ddb14b0d606c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:59.576368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3baf10-7d39-4347-9ad7-ddb14b0d606c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:59.585639 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe10b0-c7e9-40c6-b9c8-18d8e277e853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:59.586312 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe10b0-c7e9-40c6-b9c8-18d8e277e853 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:59.586595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe10b0-c7e9-40c6-b9c8-18d8e277e853 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:59.586868 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe10b0-c7e9-40c6-b9c8-18d8e277e853 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:59.587667 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe10b0-c7e9-40c6-b9c8-18d8e277e853 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:59.589130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe10b0-c7e9-40c6-b9c8-18d8e277e853 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:59.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 362/751] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:16:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:16:59.597584 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c55b1-e763-4a03-ad7e-f38790fb0b37 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:59.597883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c55b1-e763-4a03-ad7e-f38790fb0b37 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:59.598262 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c55b1-e763-4a03-ad7e-f38790fb0b37 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:59.600487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 363/752] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:16:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:16:59.601605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c55b1-e763-4a03-ad7e-f38790fb0b37 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:59.606981 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 390/753] 10.222.0.25 () {60 vars in 1141 bytes} [Mon May 9 18:16:59 2022] POST /identity/v3/services => generated 240 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:16:59.608538 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547558a-d43a-44c6-872c-b354473ed253 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:59.608824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547558a-d43a-44c6-872c-b354473ed253 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:59.609178 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547558a-d43a-44c6-872c-b354473ed253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:59.635548 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-5547558a-d43a-44c6-872c-b354473ed253 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:16:59.636816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-5547558a-d43a-44c6-872c-b354473ed253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:16:59.725697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 391/754] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:16:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:16:59.760849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547558a-d43a-44c6-872c-b354473ed253 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:16:59.773045 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c55b1-e763-4a03-ad7e-f38790fb0b37 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:16:59.848573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7913c53-7771-4a3d-b977-1a19129607e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:16:59.849660 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7913c53-7771-4a3d-b977-1a19129607e9 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:16:59.850150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7913c53-7771-4a3d-b977-1a19129607e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:16:59.850615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7913c53-7771-4a3d-b977-1a19129607e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:16:59.851697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7913c53-7771-4a3d-b977-1a19129607e9 None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=ec2db3aa2b7d41f89314fec960d898b7) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:16:59.852229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7913c53-7771-4a3d-b977-1a19129607e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:16:59.853874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7913c53-7771-4a3d-b977-1a19129607e9 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:16:59.866384 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 364/755] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:16:59 2022] GET /identity/v3/role_assignments?user.id=ec2db3aa2b7d41f89314fec960d898b7&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:16:59.895545 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547558a-d43a-44c6-872c-b354473ed253 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:00.065440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56f80a-768a-4ffa-9b6f-07b11761ec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:00.066250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56f80a-768a-4ffa-9b6f-07b11761ec32 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:00.066531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56f80a-768a-4ffa-9b6f-07b11761ec32 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:00.066803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56f80a-768a-4ffa-9b6f-07b11761ec32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:00.079036 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56f80a-768a-4ffa-9b6f-07b11761ec32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-ef56f80a-768a-4ffa-9b6f-07b11761ec32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:00.087530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:00.088483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:00.089376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:00.089376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:00.089376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:00.089376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:00.089376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:00.089376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:00.089376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:00.089376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:00.089376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:00.089376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:00.089376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:17:00.089376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:00.089376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 392/756] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:16:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:00.094617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7913c53-7771-4a3d-b977-1a19129607e9 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:00.199608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7913c53-7771-4a3d-b977-1a19129607e9 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:00.238335 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4e5e1-50ce-4852-92fa-01b1a60cd35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:00.239018 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4e5e1-50ce-4852-92fa-01b1a60cd35a None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:00.239329 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4e5e1-50ce-4852-92fa-01b1a60cd35a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:00.239604 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4e5e1-50ce-4852-92fa-01b1a60cd35a None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:00.240351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4e5e1-50ce-4852-92fa-01b1a60cd35a None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:00.240656 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4e5e1-50ce-4852-92fa-01b1a60cd35a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:00.242006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4e5e1-50ce-4852-92fa-01b1a60cd35a None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:00.255830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 365/757] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:17:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:00.261078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56f80a-768a-4ffa-9b6f-07b11761ec32 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:00.421813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56f80a-768a-4ffa-9b6f-07b11761ec32 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:00.468516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aafa7-4582-42d4-8554-fdbaf675255f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:00.469717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aafa7-4582-42d4-8554-fdbaf675255f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:00.470251 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aafa7-4582-42d4-8554-fdbaf675255f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:00.470744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aafa7-4582-42d4-8554-fdbaf675255f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:00.483741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67aafa7-4582-42d4-8554-fdbaf675255f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:00.485458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67aafa7-4582-42d4-8554-fdbaf675255f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-c67aafa7-4582-42d4-8554-fdbaf675255f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:00.509633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:00.510596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:00.511525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:00.511525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:00.511525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:00.511525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:00.513803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 393/758] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:00.520256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4e5e1-50ce-4852-92fa-01b1a60cd35a None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:00.659071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4e5e1-50ce-4852-92fa-01b1a60cd35a None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:00.706412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1275f5-f25f-4686-b1ec-56791602d4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:00.707488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1275f5-f25f-4686-b1ec-56791602d4a2 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:00.707978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1275f5-f25f-4686-b1ec-56791602d4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:00.708516 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1275f5-f25f-4686-b1ec-56791602d4a2 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:00.709531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1275f5-f25f-4686-b1ec-56791602d4a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:00.710039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1275f5-f25f-4686-b1ec-56791602d4a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:00.711674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1275f5-f25f-4686-b1ec-56791602d4a2 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:00.728477 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 366/759] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:00.733996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aafa7-4582-42d4-8554-fdbaf675255f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:00.852548 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aafa7-4582-42d4-8554-fdbaf675255f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:00.900530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88d4ed-57d3-416b-8bce-57e4ffd400e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:00.901865 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88d4ed-57d3-416b-8bce-57e4ffd400e2 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:00.902357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88d4ed-57d3-416b-8bce-57e4ffd400e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:00.902858 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88d4ed-57d3-416b-8bce-57e4ffd400e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:00.917046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c88d4ed-57d3-416b-8bce-57e4ffd400e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:00.918774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c88d4ed-57d3-416b-8bce-57e4ffd400e2 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-5c88d4ed-57d3-416b-8bce-57e4ffd400e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:17:00.936019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:00.936985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:00.937917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:00.937917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:00.937917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:17:00.937917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:17:00.937917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:17:00.937917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:17:00.937917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:17:00.937917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:17:00.937917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 18:17:00.937917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:00.944990 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 394/760] 10.222.0.25 () {56 vars in 1093 bytes} [Mon May 9 18:17:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:00.946216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1275f5-f25f-4686-b1ec-56791602d4a2 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:01.057786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1275f5-f25f-4686-b1ec-56791602d4a2 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:01.096621 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fa493-f9c9-4095-86d4-c8e7305b425b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:01.097303 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fa493-f9c9-4095-86d4-c8e7305b425b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:01.097589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fa493-f9c9-4095-86d4-c8e7305b425b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:01.097870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fa493-f9c9-4095-86d4-c8e7305b425b None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:01.098703 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fa493-f9c9-4095-86d4-c8e7305b425b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:01.099016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fa493-f9c9-4095-86d4-c8e7305b425b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:01.100458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fa493-f9c9-4095-86d4-c8e7305b425b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:01.122671 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 367/761] 10.222.0.25 () {56 vars in 1133 bytes} [Mon May 9 18:17:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:01.128578 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88d4ed-57d3-416b-8bce-57e4ffd400e2 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:01.249595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88d4ed-57d3-416b-8bce-57e4ffd400e2 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:01.293152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b83ec-9d52-4895-a298-39227d9d012a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:01.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b83ec-9d52-4895-a298-39227d9d012a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:01.294256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b83ec-9d52-4895-a298-39227d9d012a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:01.294529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b83ec-9d52-4895-a298-39227d9d012a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:01.310790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b83ec-9d52-4895-a298-39227d9d012a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:01.312175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b83ec-9d52-4895-a298-39227d9d012a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-ed0b83ec-9d52-4895-a298-39227d9d012a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:01.334453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:01.335400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:01.336290 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:01.336290 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:01.336290 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:01.336290 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:01.336290 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 395/762] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:01.343463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fa493-f9c9-4095-86d4-c8e7305b425b None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:01.456656 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fa493-f9c9-4095-86d4-c8e7305b425b None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:01.497213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64d8ef-c8fb-4775-b9d0-fe506564e571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:01.497890 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64d8ef-c8fb-4775-b9d0-fe506564e571 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:01.498174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64d8ef-c8fb-4775-b9d0-fe506564e571 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:01.498451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64d8ef-c8fb-4775-b9d0-fe506564e571 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:01.499262 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64d8ef-c8fb-4775-b9d0-fe506564e571 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:01.499571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64d8ef-c8fb-4775-b9d0-fe506564e571 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:01.500940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64d8ef-c8fb-4775-b9d0-fe506564e571 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:01.517707 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 368/763] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:01.523256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b83ec-9d52-4895-a298-39227d9d012a None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:01.685837 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b83ec-9d52-4895-a298-39227d9d012a None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:01.731557 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da022647-d626-4d00-be20-fcbddb4691c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:01.732723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da022647-d626-4d00-be20-fcbddb4691c9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:01.733205 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da022647-d626-4d00-be20-fcbddb4691c9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:01.733691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da022647-d626-4d00-be20-fcbddb4691c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:01.745519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da022647-d626-4d00-be20-fcbddb4691c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:01.747152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da022647-d626-4d00-be20-fcbddb4691c9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-da022647-d626-4d00-be20-fcbddb4691c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:17:01.765313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:01.766204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:01.767760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:17:01.767760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:17:01.767760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:17:01.767760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:17:01.767760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:17:01.767760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:17:01.767760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:01.767760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:01.770154 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 396/764] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:01.775496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64d8ef-c8fb-4775-b9d0-fe506564e571 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:01.840419 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28d3fb-b30e-4e54-84f9-8ed9a1be17fd None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:01.841059 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28d3fb-b30e-4e54-84f9-8ed9a1be17fd None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:01.841562 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28d3fb-b30e-4e54-84f9-8ed9a1be17fd None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:01.843423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 397/765] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:01.849145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aadfff-67a5-4c39-a1c5-47986034be7f None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:01.849745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aadfff-67a5-4c39-a1c5-47986034be7f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:01.850217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aadfff-67a5-4c39-a1c5-47986034be7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:01.879092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-31aadfff-67a5-4c39-a1c5-47986034be7f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:01.880293 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-31aadfff-67a5-4c39-a1c5-47986034be7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:02.018594 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 398/766] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:02.064869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aadfff-67a5-4c39-a1c5-47986034be7f None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:02.114063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64d8ef-c8fb-4775-b9d0-fe506564e571 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:02.150899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69edec5-2c4e-4c04-b29c-2174108a6ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:02.151605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69edec5-2c4e-4c04-b29c-2174108a6ee3 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:02.151888 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69edec5-2c4e-4c04-b29c-2174108a6ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:02.152164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69edec5-2c4e-4c04-b29c-2174108a6ee3 None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:02.152948 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69edec5-2c4e-4c04-b29c-2174108a6ee3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:02.153256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69edec5-2c4e-4c04-b29c-2174108a6ee3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:02.154661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69edec5-2c4e-4c04-b29c-2174108a6ee3 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:02.166686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 369/767] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:17:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:02.169137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aadfff-67a5-4c39-a1c5-47986034be7f None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:02.172655 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77d24d-736c-49a0-bff9-3d1f5dde2ef3 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:02.172920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77d24d-736c-49a0-bff9-3d1f5dde2ef3 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:02.173737 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77d24d-736c-49a0-bff9-3d1f5dde2ef3 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:02.179050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 370/768] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:02.179050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77d24d-736c-49a0-bff9-3d1f5dde2ef3 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:02.210273 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f3524-d7a7-4811-8afd-93501ca77345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:02.211498 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f3524-d7a7-4811-8afd-93501ca77345 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:02.211961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f3524-d7a7-4811-8afd-93501ca77345 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:02.212413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f3524-d7a7-4811-8afd-93501ca77345 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:02.213254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f3524-d7a7-4811-8afd-93501ca77345 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:02.214841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f3524-d7a7-4811-8afd-93501ca77345 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-794f3524-d7a7-4811-8afd-93501ca77345 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:02.227268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:02.228234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 18:17:02.229076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:02.237163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 399/769] 10.222.0.25 () {56 vars in 1129 bytes} [Mon May 9 18:17:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:02.238705 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7654b64-d484-45e8-a26c-54f7e219c611 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:02.239198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7654b64-d484-45e8-a26c-54f7e219c611 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:02.239669 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7654b64-d484-45e8-a26c-54f7e219c611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:02.257701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b7654b64-d484-45e8-a26c-54f7e219c611 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:02.259842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b7654b64-d484-45e8-a26c-54f7e219c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:02.286478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77d24d-736c-49a0-bff9-3d1f5dde2ef3 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:02.321111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cfd74-a266-481b-9a51-b3e89aa2975e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:02.322104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cfd74-a266-481b-9a51-b3e89aa2975e None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:02.322451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cfd74-a266-481b-9a51-b3e89aa2975e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:02.322811 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cfd74-a266-481b-9a51-b3e89aa2975e None admin] PATH_INFO: `/v3/projects/4849b58351c74caf92a584686c74e397/users/f769ee9f31354e5296645fb1e3bd84b6/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:02.324890 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cfd74-a266-481b-9a51-b3e89aa2975e None admin] RBAC: Authorizing `identity:create_grant(project_id=4849b58351c74caf92a584686c74e397, user_id=f769ee9f31354e5296645fb1e3bd84b6, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:02.326379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cfd74-a266-481b-9a51-b3e89aa2975e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:02.331161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 400/770] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:02.332278 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7654b64-d484-45e8-a26c-54f7e219c611 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:02.353673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 371/771] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:17:02 2022] PUT /identity/v3/projects/4849b58351c74caf92a584686c74e397/users/f769ee9f31354e5296645fb1e3bd84b6/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:17:02.374752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cfd74-a266-481b-9a51-b3e89aa2975e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:02.442852 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7654b64-d484-45e8-a26c-54f7e219c611 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:02.486195 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eec771-eea2-47e5-af1e-82251c0f6e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:02.488760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eec771-eea2-47e5-af1e-82251c0f6e4d None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:02.489601 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eec771-eea2-47e5-af1e-82251c0f6e4d None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:02.490787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eec771-eea2-47e5-af1e-82251c0f6e4d None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:02.491781 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eec771-eea2-47e5-af1e-82251c0f6e4d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:02.493094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eec771-eea2-47e5-af1e-82251c0f6e4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:02.495959 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eec771-eea2-47e5-af1e-82251c0f6e4d None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:02.505358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 401/772] 10.222.0.25 () {56 vars in 1124 bytes} [Mon May 9 18:17:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:02.512123 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eec771-eea2-47e5-af1e-82251c0f6e4d None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:02.587495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cfd74-a266-481b-9a51-b3e89aa2975e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:02.635307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eec771-eea2-47e5-af1e-82251c0f6e4d None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:02.642087 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccba62e-8ef6-4673-a3fe-a84f3fad271a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:02.643387 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccba62e-8ef6-4673-a3fe-a84f3fad271a None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:02.643700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccba62e-8ef6-4673-a3fe-a84f3fad271a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:02.643979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccba62e-8ef6-4673-a3fe-a84f3fad271a None admin] PATH_INFO: `/v3/roles/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:02.655962 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccba62e-8ef6-4673-a3fe-a84f3fad271a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-5ccba62e-8ef6-4673-a3fe-a84f3fad271a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:02.665839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:02.666803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:02.667711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:02.667711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:02.667711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:02.667711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:02.667711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:02.667711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:02.667711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:02.667711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:02.667711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:02.667711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:02.667711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:17:02.667711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:02.667711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 372/773] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:17:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:02.669238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268167bd-16d2-4cd4-8169-54b0c17f2713 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:02.669593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268167bd-16d2-4cd4-8169-54b0c17f2713 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:02.669954 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268167bd-16d2-4cd4-8169-54b0c17f2713 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:02.675203 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 373/774] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:02.675203 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268167bd-16d2-4cd4-8169-54b0c17f2713 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:02.681257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b182899-e593-42b2-93af-a290f24fc995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:02.681926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b182899-e593-42b2-93af-a290f24fc995 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:02.682207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b182899-e593-42b2-93af-a290f24fc995 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:02.682481 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b182899-e593-42b2-93af-a290f24fc995 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:02.683242 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b182899-e593-42b2-93af-a290f24fc995 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:02.683770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b182899-e593-42b2-93af-a290f24fc995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:02.685134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b182899-e593-42b2-93af-a290f24fc995 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:02.693710 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 402/775] 10.222.0.25 () {56 vars in 1124 bytes} [Mon May 9 18:17:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:02.694815 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273571c-bb2f-4393-a6f8-e7a0f51fae2c None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:02.695102 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273571c-bb2f-4393-a6f8-e7a0f51fae2c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:02.699577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273571c-bb2f-4393-a6f8-e7a0f51fae2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:02.717067 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-1273571c-bb2f-4393-a6f8-e7a0f51fae2c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:02.718229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-1273571c-bb2f-4393-a6f8-e7a0f51fae2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:02.781879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 403/776] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:02.783567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273571c-bb2f-4393-a6f8-e7a0f51fae2c None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:02.790643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268167bd-16d2-4cd4-8169-54b0c17f2713 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:02.825062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ee161-e830-4ae7-a052-15703e6bddc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:02.825738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ee161-e830-4ae7-a052-15703e6bddc4 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:02.826023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ee161-e830-4ae7-a052-15703e6bddc4 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:02.826297 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ee161-e830-4ae7-a052-15703e6bddc4 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:02.827073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ee161-e830-4ae7-a052-15703e6bddc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:02.827416 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ee161-e830-4ae7-a052-15703e6bddc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:02.828774 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ee161-e830-4ae7-a052-15703e6bddc4 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:02.837378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 374/777] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:17:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:02.838564 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04196d6b-d74c-49db-ae18-1ba90fc155c1 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:02.838883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04196d6b-d74c-49db-ae18-1ba90fc155c1 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:02.839128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04196d6b-d74c-49db-ae18-1ba90fc155c1 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:02.840496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 375/778] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:02.841028 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04196d6b-d74c-49db-ae18-1ba90fc155c1 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:02.901942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273571c-bb2f-4393-a6f8-e7a0f51fae2c None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:02.940830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04196d6b-d74c-49db-ae18-1ba90fc155c1 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:02.941652 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262df2c2-80d2-46c4-b340-2b5f0c442f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:02.942346 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262df2c2-80d2-46c4-b340-2b5f0c442f6d None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:02.942630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262df2c2-80d2-46c4-b340-2b5f0c442f6d None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:02.942900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262df2c2-80d2-46c4-b340-2b5f0c442f6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:02.943716 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262df2c2-80d2-46c4-b340-2b5f0c442f6d None admin] RBAC: Adding query filter params (interface=public, service_id=039daf4f1e45422baa23de2eba87f930, region_id=RegionOne) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:02.944030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262df2c2-80d2-46c4-b340-2b5f0c442f6d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:02.945406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262df2c2-80d2-46c4-b340-2b5f0c442f6d None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:02.957649 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 404/779] 10.222.0.25 () {56 vars in 1251 bytes} [Mon May 9 18:17:02 2022] GET /identity/v3/endpoints?service_id=039daf4f1e45422baa23de2eba87f930&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:02.957649 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262df2c2-80d2-46c4-b340-2b5f0c442f6d None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:02.991274 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e030829-41c1-4d92-9fcd-47d1fe950ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:02.992074 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e030829-41c1-4d92-9fcd-47d1fe950ca4 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:02.992351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e030829-41c1-4d92-9fcd-47d1fe950ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:02.992619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e030829-41c1-4d92-9fcd-47d1fe950ca4 None admin] PATH_INFO: `/v3/services/network` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:02.993266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e030829-41c1-4d92-9fcd-47d1fe950ca4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:02.994750 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e030829-41c1-4d92-9fcd-47d1fe950ca4 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-9e030829-41c1-4d92-9fcd-47d1fe950ca4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:03.006550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:03.007579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:03.008411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 376/780] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:02 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:03.009196 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c2624-7cf9-4c7f-8b78-36a84f6f893b None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:03.009196 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c2624-7cf9-4c7f-8b78-36a84f6f893b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:03.009879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c2624-7cf9-4c7f-8b78-36a84f6f893b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:03.026495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-498c2624-7cf9-4c7f-8b78-36a84f6f893b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:03.027757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-498c2624-7cf9-4c7f-8b78-36a84f6f893b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:03.058111 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262df2c2-80d2-46c4-b340-2b5f0c442f6d None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:03.091883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 377/781] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:03.092823 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c2624-7cf9-4c7f-8b78-36a84f6f893b None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:03.093329 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7a4d3-c418-4eb6-abde-a7fd67abe63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:03.094321 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7a4d3-c418-4eb6-abde-a7fd67abe63a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:03.094781 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7a4d3-c418-4eb6-abde-a7fd67abe63a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:03.095807 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7a4d3-c418-4eb6-abde-a7fd67abe63a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:03.111581 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7a4d3-c418-4eb6-abde-a7fd67abe63a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:03.112844 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7a4d3-c418-4eb6-abde-a7fd67abe63a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-14c7a4d3-c418-4eb6-abde-a7fd67abe63a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:03.129756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:03.130657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:03.131972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:03.131972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:03.131972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:03.131972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:03.131972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 405/782] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:03.132278 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7a4d3-c418-4eb6-abde-a7fd67abe63a None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:03.194312 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c2624-7cf9-4c7f-8b78-36a84f6f893b None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:03.227859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99bb83-7f07-458e-8c42-d456b61c097f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:03.228553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99bb83-7f07-458e-8c42-d456b61c097f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:03.228833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99bb83-7f07-458e-8c42-d456b61c097f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:03.229056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7a4d3-c418-4eb6-abde-a7fd67abe63a None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:03.229513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99bb83-7f07-458e-8c42-d456b61c097f None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:03.229769 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99bb83-7f07-458e-8c42-d456b61c097f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:03.231111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99bb83-7f07-458e-8c42-d456b61c097f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:03.236374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 378/783] 10.222.0.25 () {56 vars in 1087 bytes} [Mon May 9 18:17:03 2022] GET /identity/v3/services => generated 772 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:03.236953 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99bb83-7f07-458e-8c42-d456b61c097f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:03.276829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1afe1-fe7d-4691-a80c-5cdd8f1fb8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:03.277894 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1afe1-fe7d-4691-a80c-5cdd8f1fb8f9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:03.278559 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1afe1-fe7d-4691-a80c-5cdd8f1fb8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:03.279972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1afe1-fe7d-4691-a80c-5cdd8f1fb8f9 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:03.281054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1afe1-fe7d-4691-a80c-5cdd8f1fb8f9 None admin] RBAC: Adding query filter params (name=network) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:03.281572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1afe1-fe7d-4691-a80c-5cdd8f1fb8f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:03.283329 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1afe1-fe7d-4691-a80c-5cdd8f1fb8f9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:03.297043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 406/784] 10.222.0.25 () {56 vars in 1112 bytes} [Mon May 9 18:17:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:03.297704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1afe1-fe7d-4691-a80c-5cdd8f1fb8f9 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:03.359263 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99bb83-7f07-458e-8c42-d456b61c097f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:03.395918 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646d3bc-5807-4d57-97a6-eda3b6b2765c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:03.396736 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646d3bc-5807-4d57-97a6-eda3b6b2765c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:03.397009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646d3bc-5807-4d57-97a6-eda3b6b2765c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:03.397272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646d3bc-5807-4d57-97a6-eda3b6b2765c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:03.408737 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646d3bc-5807-4d57-97a6-eda3b6b2765c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:03.415819 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1afe1-fe7d-4691-a80c-5cdd8f1fb8f9 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-6646d3bc-5807-4d57-97a6-eda3b6b2765c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:03.416514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:03.417444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:03.418684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:03.418684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:03.418684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:03.418684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:03.418684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:03.418684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:03.418684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:03.418684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:03.418684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:03.418684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:03.418684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:17:03.418684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:03.418684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 379/785] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:17:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:03.419422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646d3bc-5807-4d57-97a6-eda3b6b2765c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:03.457721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd2e01-2e5f-473d-b4db-c5905f83bd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:03.458391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd2e01-2e5f-473d-b4db-c5905f83bd6e None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:03.458670 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd2e01-2e5f-473d-b4db-c5905f83bd6e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:03.458942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd2e01-2e5f-473d-b4db-c5905f83bd6e None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:03.459689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd2e01-2e5f-473d-b4db-c5905f83bd6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:03.460669 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd2e01-2e5f-473d-b4db-c5905f83bd6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:03.461332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd2e01-2e5f-473d-b4db-c5905f83bd6e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:03.479223 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 407/786] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:03.480163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd2e01-2e5f-473d-b4db-c5905f83bd6e None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:03.534033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646d3bc-5807-4d57-97a6-eda3b6b2765c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:03.576593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60fb02-605d-4c4b-afc1-476340b1661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:03.577574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60fb02-605d-4c4b-afc1-476340b1661c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:03.578118 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60fb02-605d-4c4b-afc1-476340b1661c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:03.578398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60fb02-605d-4c4b-afc1-476340b1661c None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:03.579106 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60fb02-605d-4c4b-afc1-476340b1661c None admin] RBAC: Adding query filter params (type=network) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:03.579989 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60fb02-605d-4c4b-afc1-476340b1661c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:03.580517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd2e01-2e5f-473d-b4db-c5905f83bd6e None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:03.581352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60fb02-605d-4c4b-afc1-476340b1661c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:03.594291 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 380/787] 10.222.0.25 () {56 vars in 1112 bytes} [Mon May 9 18:17:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:03.595291 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60fb02-605d-4c4b-afc1-476340b1661c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:03.625269 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bb690-9103-4008-b729-f96449ca9dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:03.626020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bb690-9103-4008-b729-f96449ca9dfd None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:03.626856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bb690-9103-4008-b729-f96449ca9dfd None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:03.627387 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bb690-9103-4008-b729-f96449ca9dfd None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:03.628147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bb690-9103-4008-b729-f96449ca9dfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:03.628449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bb690-9103-4008-b729-f96449ca9dfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:03.629809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bb690-9103-4008-b729-f96449ca9dfd None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:03.647701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 408/788] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:17:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:03.648644 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bb690-9103-4008-b729-f96449ca9dfd None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:03.708423 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60fb02-605d-4c4b-afc1-476340b1661c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:03.743524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65ffe9-ee11-47a6-9f9e-356fa8858f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:03.744443 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65ffe9-ee11-47a6-9f9e-356fa8858f47 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:03.744800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65ffe9-ee11-47a6-9f9e-356fa8858f47 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:03.745149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65ffe9-ee11-47a6-9f9e-356fa8858f47 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:03.754288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65ffe9-ee11-47a6-9f9e-356fa8858f47 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:03.755844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65ffe9-ee11-47a6-9f9e-356fa8858f47 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-7d65ffe9-ee11-47a6-9f9e-356fa8858f47 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:17:03.770980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:03.773108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:03.775340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:03.775340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:03.775340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:17:03.775340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:17:03.775340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:17:03.775340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:17:03.775340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:17:03.775340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:17:03.775340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 18:17:03.775340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:03.776557 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bb690-9103-4008-b729-f96449ca9dfd None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:03.783216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 381/789] 10.222.0.25 () {56 vars in 1117 bytes} [Mon May 9 18:17:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:03.783216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65ffe9-ee11-47a6-9f9e-356fa8858f47 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:03.806214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d4677-a2b1-40a7-882e-ed3318684722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:03.806875 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4677-a2b1-40a7-882e-ed3318684722 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:03.807157 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4677-a2b1-40a7-882e-ed3318684722 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:03.807456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4677-a2b1-40a7-882e-ed3318684722 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:03.808237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d4677-a2b1-40a7-882e-ed3318684722 None admin] RBAC: Adding query filter params (interface=public, service_id=bbc78f35ecdc432daa04125d46379363, region_id=RegionOne) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:03.808540 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d4677-a2b1-40a7-882e-ed3318684722 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:03.809892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d4677-a2b1-40a7-882e-ed3318684722 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:03.816661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 409/790] 10.222.0.25 () {56 vars in 1251 bytes} [Mon May 9 18:17:03 2022] GET /identity/v3/endpoints?service_id=bbc78f35ecdc432daa04125d46379363&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:03.816661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d4677-a2b1-40a7-882e-ed3318684722 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:03.886427 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65ffe9-ee11-47a6-9f9e-356fa8858f47 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:03.913745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d4677-a2b1-40a7-882e-ed3318684722 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:03.924909 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c442b9a-fac9-4cc5-8abd-0d588ad8ca56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:03.926124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c442b9a-fac9-4cc5-8abd-0d588ad8ca56 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:03.926621 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c442b9a-fac9-4cc5-8abd-0d588ad8ca56 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:03.927134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c442b9a-fac9-4cc5-8abd-0d588ad8ca56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:03.939397 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c442b9a-fac9-4cc5-8abd-0d588ad8ca56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:03.940853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c442b9a-fac9-4cc5-8abd-0d588ad8ca56 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:03.951274 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3654c4-b221-4c5f-bf59-4d5ef8289236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:03.951944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3654c4-b221-4c5f-bf59-4d5ef8289236 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:03.952220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3654c4-b221-4c5f-bf59-4d5ef8289236 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:03.952492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3654c4-b221-4c5f-bf59-4d5ef8289236 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:03.953322 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3654c4-b221-4c5f-bf59-4d5ef8289236 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:03.953625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3654c4-b221-4c5f-bf59-4d5ef8289236 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:03.955084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3654c4-b221-4c5f-bf59-4d5ef8289236 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-3c442b9a-fac9-4cc5-8abd-0d588ad8ca56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:03.961824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:03.963506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:03.977001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:03.977001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:03.977001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:03.977001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:03.977001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 382/791] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:03.977001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c442b9a-fac9-4cc5-8abd-0d588ad8ca56 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:03.977853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 410/792] 10.222.0.25 () {56 vars in 1149 bytes} [Mon May 9 18:17:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:03.983220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3654c4-b221-4c5f-bf59-4d5ef8289236 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:04.070032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c442b9a-fac9-4cc5-8abd-0d588ad8ca56 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:04.076393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3654c4-b221-4c5f-bf59-4d5ef8289236 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:04.103947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405211d-808f-4d12-9f65-367c93385831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:04.104620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405211d-808f-4d12-9f65-367c93385831 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:04.104903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405211d-808f-4d12-9f65-367c93385831 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:04.105182 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405211d-808f-4d12-9f65-367c93385831 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:04.105857 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405211d-808f-4d12-9f65-367c93385831 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:04.107327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405211d-808f-4d12-9f65-367c93385831 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:04.110383 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58a0ea-e4f3-42ad-82ab-c3f4a6818e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:04.111722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58a0ea-e4f3-42ad-82ab-c3f4a6818e9f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:04.112069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58a0ea-e4f3-42ad-82ab-c3f4a6818e9f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:04.112404 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58a0ea-e4f3-42ad-82ab-c3f4a6818e9f None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:04.113180 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58a0ea-e4f3-42ad-82ab-c3f4a6818e9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:04.113547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58a0ea-e4f3-42ad-82ab-c3f4a6818e9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:04.114936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58a0ea-e4f3-42ad-82ab-c3f4a6818e9f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:04.119328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 383/793] 10.222.0.25 () {56 vars in 1087 bytes} [Mon May 9 18:17:03 2022] GET /identity/v3/services => generated 772 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:04.120429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405211d-808f-4d12-9f65-367c93385831 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:04.131323 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 411/794] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:04.133621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58a0ea-e4f3-42ad-82ab-c3f4a6818e9f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:04.272193 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58a0ea-e4f3-42ad-82ab-c3f4a6818e9f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:04.308423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633bec6-7703-4cbf-991e-51d7329b81fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:04.309237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633bec6-7703-4cbf-991e-51d7329b81fd None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:04.309517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633bec6-7703-4cbf-991e-51d7329b81fd None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:04.309790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633bec6-7703-4cbf-991e-51d7329b81fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:04.318462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633bec6-7703-4cbf-991e-51d7329b81fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:04.319931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633bec6-7703-4cbf-991e-51d7329b81fd None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-0633bec6-7703-4cbf-991e-51d7329b81fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:17:04.335751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:04.338031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:04.340261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:04.340261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:04.340261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:17:04.340261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:17:04.340261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:17:04.340261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:17:04.340261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:17:04.340261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:17:04.340261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 18:17:04.340261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:04.340261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 412/795] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:17:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:04.341641 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405211d-808f-4d12-9f65-367c93385831 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:04.347252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633bec6-7703-4cbf-991e-51d7329b81fd None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:04.388367 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b285f2-aacc-4fde-bcde-104e35ce525a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:04.389358 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b285f2-aacc-4fde-bcde-104e35ce525a None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:04.389642 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b285f2-aacc-4fde-bcde-104e35ce525a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:04.390725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b285f2-aacc-4fde-bcde-104e35ce525a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:04.404907 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b285f2-aacc-4fde-bcde-104e35ce525a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:04.406662 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b285f2-aacc-4fde-bcde-104e35ce525a None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-f6b285f2-aacc-4fde-bcde-104e35ce525a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:04.429727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:04.431849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:04.433485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:04.433485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:04.433485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:04.433485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:04.433485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 384/796] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:04.440253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b285f2-aacc-4fde-bcde-104e35ce525a None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:04.468964 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633bec6-7703-4cbf-991e-51d7329b81fd None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:04.509247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bd064-7473-45ed-8cf9-b98b00e78101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:04.509247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bd064-7473-45ed-8cf9-b98b00e78101 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:04.509247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bd064-7473-45ed-8cf9-b98b00e78101 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:04.509247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bd064-7473-45ed-8cf9-b98b00e78101 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:04.510172 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851bd064-7473-45ed-8cf9-b98b00e78101 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:04.510172 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851bd064-7473-45ed-8cf9-b98b00e78101 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:04.511225 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851bd064-7473-45ed-8cf9-b98b00e78101 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:04.535638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 413/797] 10.222.0.25 () {56 vars in 1135 bytes} [Mon May 9 18:17:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:04.547268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bd064-7473-45ed-8cf9-b98b00e78101 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:04.600400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b285f2-aacc-4fde-bcde-104e35ce525a None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:04.675042 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e9ec2-a45a-4619-8cea-0f2ecdda01ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:04.675747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e9ec2-a45a-4619-8cea-0f2ecdda01ce None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:04.676033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e9ec2-a45a-4619-8cea-0f2ecdda01ce None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:04.676468 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e9ec2-a45a-4619-8cea-0f2ecdda01ce None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:04.677203 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e9ec2-a45a-4619-8cea-0f2ecdda01ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:04.677510 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e9ec2-a45a-4619-8cea-0f2ecdda01ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:04.678881 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e9ec2-a45a-4619-8cea-0f2ecdda01ce None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:04.706079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 385/798] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:04.711426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e9ec2-a45a-4619-8cea-0f2ecdda01ce None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:04.716302 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bd064-7473-45ed-8cf9-b98b00e78101 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:04.761312 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426ae08-4336-44f0-aef8-6d5421d20021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:04.762152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426ae08-4336-44f0-aef8-6d5421d20021 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:04.762438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426ae08-4336-44f0-aef8-6d5421d20021 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:04.762712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426ae08-4336-44f0-aef8-6d5421d20021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:04.777375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426ae08-4336-44f0-aef8-6d5421d20021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:04.778809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426ae08-4336-44f0-aef8-6d5421d20021 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-1426ae08-4336-44f0-aef8-6d5421d20021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:04.807903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:04.808824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:04.809726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:04.809726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:04.809726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:04.809726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:04.809726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 414/799] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:04.815198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768610bc-b9ab-45ca-8399-f85bda86d655 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:04.815198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768610bc-b9ab-45ca-8399-f85bda86d655 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:04.815198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768610bc-b9ab-45ca-8399-f85bda86d655 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:04.815198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 415/800] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:04.815198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768610bc-b9ab-45ca-8399-f85bda86d655 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:04.852229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e9ec2-a45a-4619-8cea-0f2ecdda01ce None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:04.895543 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296a57a-8932-4e25-8f65-7f187200b14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:04.896391 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296a57a-8932-4e25-8f65-7f187200b14c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:04.896677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296a57a-8932-4e25-8f65-7f187200b14c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:04.896948 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296a57a-8932-4e25-8f65-7f187200b14c None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:04.903814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f296a57a-8932-4e25-8f65-7f187200b14c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:04.905230 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f296a57a-8932-4e25-8f65-7f187200b14c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-f296a57a-8932-4e25-8f65-7f187200b14c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:17:04.919052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:04.921039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:04.922933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:17:04.922933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:17:04.922933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:17:04.922933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:17:04.922933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:17:04.922933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:17:04.922933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:04.922933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:04.923890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768610bc-b9ab-45ca-8399-f85bda86d655 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:04.931212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 386/801] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:04.931212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb05f33-a82b-4457-a0fb-128bd07d4221 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:04.931212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb05f33-a82b-4457-a0fb-128bd07d4221 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:04.931212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb05f33-a82b-4457-a0fb-128bd07d4221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:04.944230 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-3fb05f33-a82b-4457-a0fb-128bd07d4221 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:04.945409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-3fb05f33-a82b-4457-a0fb-128bd07d4221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:04.966283 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7a1f9-563f-4e3c-8426-558663f43322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:04.966283 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7a1f9-563f-4e3c-8426-558663f43322 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:04.966283 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7a1f9-563f-4e3c-8426-558663f43322 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:04.966283 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7a1f9-563f-4e3c-8426-558663f43322 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:04.968119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7a1f9-563f-4e3c-8426-558663f43322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:04.968119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7a1f9-563f-4e3c-8426-558663f43322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:04.968790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7a1f9-563f-4e3c-8426-558663f43322 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:04.984268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 416/802] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:04.985182 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7a1f9-563f-4e3c-8426-558663f43322 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:05.015949 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 387/803] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:05.023695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb05f33-a82b-4457-a0fb-128bd07d4221 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:05.103491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7a1f9-563f-4e3c-8426-558663f43322 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:05.137677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb05f33-a82b-4457-a0fb-128bd07d4221 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:05.146087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4334f-5b48-4199-af30-f52ff2026737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:05.146755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4334f-5b48-4199-af30-f52ff2026737 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:05.147041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4334f-5b48-4199-af30-f52ff2026737 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:05.147366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4334f-5b48-4199-af30-f52ff2026737 None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:05.148505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4334f-5b48-4199-af30-f52ff2026737 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:05.148831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4334f-5b48-4199-af30-f52ff2026737 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:05.150599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4334f-5b48-4199-af30-f52ff2026737 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:05.170698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 417/804] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:17:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:05.170698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4334f-5b48-4199-af30-f52ff2026737 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:05.179582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953e7ba-3646-4f43-a9d3-36622790a4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:05.180738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953e7ba-3646-4f43-a9d3-36622790a4e7 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:05.181245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953e7ba-3646-4f43-a9d3-36622790a4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:05.181743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953e7ba-3646-4f43-a9d3-36622790a4e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:05.189903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953e7ba-3646-4f43-a9d3-36622790a4e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:05.191717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953e7ba-3646-4f43-a9d3-36622790a4e7 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-8953e7ba-3646-4f43-a9d3-36622790a4e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:17:05.206111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:05.207072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:05.207986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:17:05.207986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:17:05.207986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:17:05.207986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:17:05.207986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:17:05.207986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:17:05.207986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:05.207986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:05.210537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 388/805] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:05.211621 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953e7ba-3646-4f43-a9d3-36622790a4e7 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:05.287100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4334f-5b48-4199-af30-f52ff2026737 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:05.316493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953e7ba-3646-4f43-a9d3-36622790a4e7 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:05.332047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab21e9-284f-44ab-a2ec-7e4e26844199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:05.332997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab21e9-284f-44ab-a2ec-7e4e26844199 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:05.333112 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab21e9-284f-44ab-a2ec-7e4e26844199 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:05.333393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab21e9-284f-44ab-a2ec-7e4e26844199 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:05.343519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab21e9-284f-44ab-a2ec-7e4e26844199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-3aab21e9-284f-44ab-a2ec-7e4e26844199 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:05.351577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:05.352505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:05.353333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:05.353333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:05.353333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:05.353333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:05.353333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:05.353333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:05.353333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:05.353333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:05.353333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:05.353333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:05.353333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:17:05.353333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:05.353333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 418/806] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:17:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:05.354087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab21e9-284f-44ab-a2ec-7e4e26844199 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:05.359722 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb020e-5480-4996-b4f3-cc2726e02f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:05.359722 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb020e-5480-4996-b4f3-cc2726e02f92 None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:05.359722 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb020e-5480-4996-b4f3-cc2726e02f92 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:05.359722 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb020e-5480-4996-b4f3-cc2726e02f92 None admin] PATH_INFO: `/v3/projects/4849b58351c74caf92a584686c74e397/users/154f86b7575e47efb2a5e1c35943e037/roles/7b2e521b45dc4a98adad996ca0027de6` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:05.379266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb020e-5480-4996-b4f3-cc2726e02f92 None admin] RBAC: Authorizing `identity:create_grant(project_id=4849b58351c74caf92a584686c74e397, user_id=154f86b7575e47efb2a5e1c35943e037, role_id=7b2e521b45dc4a98adad996ca0027de6)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:05.381006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb020e-5480-4996-b4f3-cc2726e02f92 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:05.403789 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 389/807] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:17:05 2022] PUT /identity/v3/projects/4849b58351c74caf92a584686c74e397/users/154f86b7575e47efb2a5e1c35943e037/roles/7b2e521b45dc4a98adad996ca0027de6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:17:05.403789 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb020e-5480-4996-b4f3-cc2726e02f92 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:05.535059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb020e-5480-4996-b4f3-cc2726e02f92 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:05.557441 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab21e9-284f-44ab-a2ec-7e4e26844199 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:05.581396 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553db5a2-0372-4d36-81e8-03d93c59823e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:05.582458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553db5a2-0372-4d36-81e8-03d93c59823e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:05.583024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553db5a2-0372-4d36-81e8-03d93c59823e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:05.583530 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553db5a2-0372-4d36-81e8-03d93c59823e None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:05.584580 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553db5a2-0372-4d36-81e8-03d93c59823e None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:05.585077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553db5a2-0372-4d36-81e8-03d93c59823e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:05.586661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553db5a2-0372-4d36-81e8-03d93c59823e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:05.596608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 390/808] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:17:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:05.602051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553db5a2-0372-4d36-81e8-03d93c59823e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:05.625508 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d36c67-54eb-4ba7-80ef-68b2e39fc666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:05.626514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d36c67-54eb-4ba7-80ef-68b2e39fc666 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:05.626980 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d36c67-54eb-4ba7-80ef-68b2e39fc666 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:05.627705 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d36c67-54eb-4ba7-80ef-68b2e39fc666 None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:05.629451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d36c67-54eb-4ba7-80ef-68b2e39fc666 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:05.630153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d36c67-54eb-4ba7-80ef-68b2e39fc666 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:05.632718 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d36c67-54eb-4ba7-80ef-68b2e39fc666 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:05.647948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 419/809] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:17:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:05.654760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d36c67-54eb-4ba7-80ef-68b2e39fc666 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:05.743730 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553db5a2-0372-4d36-81e8-03d93c59823e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:05.778353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2ac69-ad83-4104-94e4-e61c6df18771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:05.779970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2ac69-ad83-4104-94e4-e61c6df18771 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:05.780587 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2ac69-ad83-4104-94e4-e61c6df18771 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:05.781152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2ac69-ad83-4104-94e4-e61c6df18771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:05.786359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d36c67-54eb-4ba7-80ef-68b2e39fc666 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:05.795359 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2ac69-ad83-4104-94e4-e61c6df18771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:05.797476 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2ac69-ad83-4104-94e4-e61c6df18771 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-6bc2ac69-ad83-4104-94e4-e61c6df18771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:05.822440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:05.823475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:05.824349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:05.824349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:05.824349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:05.824349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:05.825251 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 391/810] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:05.826064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee384-3c51-4c88-bf35-b49b1f135e41 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:05.826346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee384-3c51-4c88-bf35-b49b1f135e41 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:05.826620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee384-3c51-4c88-bf35-b49b1f135e41 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:05.828119 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 392/811] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:05.832019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcee384-3c51-4c88-bf35-b49b1f135e41 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:05.832933 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696e4b4-5b84-403c-8d49-8703a74108b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:05.834121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696e4b4-5b84-403c-8d49-8703a74108b2 None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:05.834605 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696e4b4-5b84-403c-8d49-8703a74108b2 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:05.835086 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696e4b4-5b84-403c-8d49-8703a74108b2 None admin] PATH_INFO: `/v3/projects/4849b58351c74caf92a584686c74e397/users/ec2db3aa2b7d41f89314fec960d898b7/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:05.837435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696e4b4-5b84-403c-8d49-8703a74108b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4849b58351c74caf92a584686c74e397, user_id=ec2db3aa2b7d41f89314fec960d898b7, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:05.839169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696e4b4-5b84-403c-8d49-8703a74108b2 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:05.856393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 420/812] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:17:05 2022] PUT /identity/v3/projects/4849b58351c74caf92a584686c74e397/users/ec2db3aa2b7d41f89314fec960d898b7/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:17:05.858007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3832f-ee8c-41de-a8fc-7e5bb4e6f03f None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:05.858565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3832f-ee8c-41de-a8fc-7e5bb4e6f03f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:05.859075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3832f-ee8c-41de-a8fc-7e5bb4e6f03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:05.879165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-7bf3832f-ee8c-41de-a8fc-7e5bb4e6f03f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:05.881491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7bf3832f-ee8c-41de-a8fc-7e5bb4e6f03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:05.963428 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 421/813] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:05.997680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3832f-ee8c-41de-a8fc-7e5bb4e6f03f None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:06.072008 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcee384-3c51-4c88-bf35-b49b1f135e41 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:06.118217 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92acf8-95a3-45e6-8ef8-186ce382731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:06.119366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92acf8-95a3-45e6-8ef8-186ce382731c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:06.119873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92acf8-95a3-45e6-8ef8-186ce382731c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:06.120353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92acf8-95a3-45e6-8ef8-186ce382731c None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:06.121276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92acf8-95a3-45e6-8ef8-186ce382731c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:06.121784 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92acf8-95a3-45e6-8ef8-186ce382731c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:06.123368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92acf8-95a3-45e6-8ef8-186ce382731c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:06.136592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 393/814] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:06.144179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92acf8-95a3-45e6-8ef8-186ce382731c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:06.158026 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3832f-ee8c-41de-a8fc-7e5bb4e6f03f None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:06.192614 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664375bd-bb9e-4def-9448-cdbe0cb77a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:06.193428 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664375bd-bb9e-4def-9448-cdbe0cb77a48 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:06.193708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664375bd-bb9e-4def-9448-cdbe0cb77a48 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:06.193980 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664375bd-bb9e-4def-9448-cdbe0cb77a48 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:06.194699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664375bd-bb9e-4def-9448-cdbe0cb77a48 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:06.195995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664375bd-bb9e-4def-9448-cdbe0cb77a48 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-664375bd-bb9e-4def-9448-cdbe0cb77a48 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:06.209674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:06.210675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:06.211976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 422/815] 10.222.0.25 () {56 vars in 1129 bytes} [Mon May 9 18:17:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:06.217803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664375bd-bb9e-4def-9448-cdbe0cb77a48 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:06.251580 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92acf8-95a3-45e6-8ef8-186ce382731c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:06.293022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa5ac6-fff5-4684-9fad-7adf1f6ef8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:06.294210 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa5ac6-fff5-4684-9fad-7adf1f6ef8ab None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:06.294699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa5ac6-fff5-4684-9fad-7adf1f6ef8ab None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:06.295208 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa5ac6-fff5-4684-9fad-7adf1f6ef8ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:06.303087 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa5ac6-fff5-4684-9fad-7adf1f6ef8ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:06.304835 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa5ac6-fff5-4684-9fad-7adf1f6ef8ab None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-31aa5ac6-fff5-4684-9fad-7adf1f6ef8ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:17:06.320164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:06.321174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:06.322107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:06.322107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:06.322107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:17:06.322107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:17:06.322107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:17:06.322107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:17:06.322107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:17:06.322107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:17:06.322107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 18:17:06.322107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:06.324770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 394/816] 10.222.0.25 () {56 vars in 1093 bytes} [Mon May 9 18:17:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:06.331165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa5ac6-fff5-4684-9fad-7adf1f6ef8ab None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:06.417417 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664375bd-bb9e-4def-9448-cdbe0cb77a48 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:06.432174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa5ac6-fff5-4684-9fad-7adf1f6ef8ab None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:06.467340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362f08d-c955-4cc4-9b57-44d431851fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:06.468381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362f08d-c955-4cc4-9b57-44d431851fc3 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:06.468899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362f08d-c955-4cc4-9b57-44d431851fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:06.469374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362f08d-c955-4cc4-9b57-44d431851fc3 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:06.471452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362f08d-c955-4cc4-9b57-44d431851fc3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:06.472316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362f08d-c955-4cc4-9b57-44d431851fc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:06.474602 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362f08d-c955-4cc4-9b57-44d431851fc3 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:06.495280 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 395/817] 10.222.0.25 () {56 vars in 1133 bytes} [Mon May 9 18:17:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:06.500415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362f08d-c955-4cc4-9b57-44d431851fc3 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:06.508793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd631a-7fc4-4a3b-8f67-5e1a262a6f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:06.509470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd631a-7fc4-4a3b-8f67-5e1a262a6f49 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:06.509758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd631a-7fc4-4a3b-8f67-5e1a262a6f49 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:06.510038 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd631a-7fc4-4a3b-8f67-5e1a262a6f49 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:06.510796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd631a-7fc4-4a3b-8f67-5e1a262a6f49 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:06.511046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd631a-7fc4-4a3b-8f67-5e1a262a6f49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:06.512440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd631a-7fc4-4a3b-8f67-5e1a262a6f49 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:06.524868 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 423/818] 10.222.0.25 () {56 vars in 1124 bytes} [Mon May 9 18:17:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:06.529774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd631a-7fc4-4a3b-8f67-5e1a262a6f49 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:06.616993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362f08d-c955-4cc4-9b57-44d431851fc3 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:06.656792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b95a89-a571-4acb-ae81-35023ad21166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:06.658429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b95a89-a571-4acb-ae81-35023ad21166 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:06.658947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b95a89-a571-4acb-ae81-35023ad21166 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:06.659513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b95a89-a571-4acb-ae81-35023ad21166 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:06.760534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b95a89-a571-4acb-ae81-35023ad21166 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:06.761985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b95a89-a571-4acb-ae81-35023ad21166 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:06.765769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd631a-7fc4-4a3b-8f67-5e1a262a6f49 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:06.806281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d668def-081b-40b8-b42b-f3c206e533b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:06.806948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d668def-081b-40b8-b42b-f3c206e533b2 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:06.807250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d668def-081b-40b8-b42b-f3c206e533b2 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:06.807535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d668def-081b-40b8-b42b-f3c206e533b2 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:06.808229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d668def-081b-40b8-b42b-f3c206e533b2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:06.808534 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d668def-081b-40b8-b42b-f3c206e533b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:06.809904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d668def-081b-40b8-b42b-f3c206e533b2 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:06.818081 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 424/819] 10.222.0.25 () {56 vars in 1124 bytes} [Mon May 9 18:17:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:06.823995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d668def-081b-40b8-b42b-f3c206e533b2 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:06.924962 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 396/820] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:17:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:17:06.926346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd5ad8-87b7-44eb-98ac-a408113d99b2 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:06.926637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd5ad8-87b7-44eb-98ac-a408113d99b2 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:06.926916 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd5ad8-87b7-44eb-98ac-a408113d99b2 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:06.928528 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 397/821] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:06.929910 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd5ad8-87b7-44eb-98ac-a408113d99b2 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:06.972738 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d668def-081b-40b8-b42b-f3c206e533b2 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:07.014185 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8ec06-714b-433a-b28d-7b2f1885c0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:07.014756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8ec06-714b-433a-b28d-7b2f1885c0d4 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:07.015034 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8ec06-714b-433a-b28d-7b2f1885c0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:07.015334 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8ec06-714b-433a-b28d-7b2f1885c0d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:07.016143 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8ec06-714b-433a-b28d-7b2f1885c0d4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:07.018095 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8ec06-714b-433a-b28d-7b2f1885c0d4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:07.033013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd5ad8-87b7-44eb-98ac-a408113d99b2 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:07.038082 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 425/822] 10.222.0.25 () {60 vars in 1144 bytes} [Mon May 9 18:17:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:17:07.039627 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db7ddc-897f-4b9a-ab11-33bdef31f3e5 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:07.039904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db7ddc-897f-4b9a-ab11-33bdef31f3e5 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:07.040179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db7ddc-897f-4b9a-ab11-33bdef31f3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:07.056634 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-39db7ddc-897f-4b9a-ab11-33bdef31f3e5 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:07.057837 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-39db7ddc-897f-4b9a-ab11-33bdef31f3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:07.119982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8d416-1b77-44b1-a2d3-4d11b279a07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:07.120796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8d416-1b77-44b1-a2d3-4d11b279a07f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:07.121440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8d416-1b77-44b1-a2d3-4d11b279a07f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:07.121440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8d416-1b77-44b1-a2d3-4d11b279a07f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:07.130547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 426/823] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:07.144047 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8d416-1b77-44b1-a2d3-4d11b279a07f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:07.145511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8d416-1b77-44b1-a2d3-4d11b279a07f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-cba8d416-1b77-44b1-a2d3-4d11b279a07f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:07.169585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:07.171931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:07.171931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:07.171931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:07.171931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:07.171931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 398/824] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:07.178426 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db7ddc-897f-4b9a-ab11-33bdef31f3e5 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:07.191002 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8d416-1b77-44b1-a2d3-4d11b279a07f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:07.312777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db7ddc-897f-4b9a-ab11-33bdef31f3e5 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:07.352712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1bcab-453a-43fb-a866-7a0d00bfd69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:07.353508 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1bcab-453a-43fb-a866-7a0d00bfd69a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:07.353782 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1bcab-453a-43fb-a866-7a0d00bfd69a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:07.354052 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1bcab-453a-43fb-a866-7a0d00bfd69a None admin] PATH_INFO: `/v3/services/network` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:07.354695 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1bcab-453a-43fb-a866-7a0d00bfd69a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:07.356088 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1bcab-453a-43fb-a866-7a0d00bfd69a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:07.356951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8d416-1b77-44b1-a2d3-4d11b279a07f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-b5e1bcab-453a-43fb-a866-7a0d00bfd69a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:07.368416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:07.370830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:07.372870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 427/825] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:07 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:07.377005 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1bcab-453a-43fb-a866-7a0d00bfd69a None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:07.408721 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f12b3-2b2a-4f8c-b418-f7d6cd6cc458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:07.409407 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f12b3-2b2a-4f8c-b418-f7d6cd6cc458 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:07.409691 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f12b3-2b2a-4f8c-b418-f7d6cd6cc458 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:07.409967 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f12b3-2b2a-4f8c-b418-f7d6cd6cc458 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:07.410961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f12b3-2b2a-4f8c-b418-f7d6cd6cc458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:07.411299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f12b3-2b2a-4f8c-b418-f7d6cd6cc458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:07.413125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f12b3-2b2a-4f8c-b418-f7d6cd6cc458 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:07.428879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 399/826] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:07.434233 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f12b3-2b2a-4f8c-b418-f7d6cd6cc458 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:07.508223 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1bcab-453a-43fb-a866-7a0d00bfd69a None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:07.543325 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce7f34-8d8b-462e-a3cc-602c2e1284ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:07.544653 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce7f34-8d8b-462e-a3cc-602c2e1284ef None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:07.545133 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce7f34-8d8b-462e-a3cc-602c2e1284ef None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:07.545758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce7f34-8d8b-462e-a3cc-602c2e1284ef None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:07.547013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce7f34-8d8b-462e-a3cc-602c2e1284ef None admin] RBAC: Adding query filter params (name=network) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:07.547722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce7f34-8d8b-462e-a3cc-602c2e1284ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:07.549860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce7f34-8d8b-462e-a3cc-602c2e1284ef None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:07.555627 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 428/827] 10.222.0.25 () {56 vars in 1112 bytes} [Mon May 9 18:17:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:07.564912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce7f34-8d8b-462e-a3cc-602c2e1284ef None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:07.621749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f12b3-2b2a-4f8c-b418-f7d6cd6cc458 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:07.692273 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9431f5-0929-4338-8912-f74ba9d8177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:07.693092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9431f5-0929-4338-8912-f74ba9d8177f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:07.693378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9431f5-0929-4338-8912-f74ba9d8177f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:07.693651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9431f5-0929-4338-8912-f74ba9d8177f None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:07.704917 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9431f5-0929-4338-8912-f74ba9d8177f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:07.706359 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9431f5-0929-4338-8912-f74ba9d8177f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-fa9431f5-0929-4338-8912-f74ba9d8177f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:17:07.731527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:07.732469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:07.733349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:17:07.733349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:17:07.733349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:17:07.733349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:17:07.733349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:17:07.733349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:17:07.733349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:07.733349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:07.736940 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce7f34-8d8b-462e-a3cc-602c2e1284ef None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:07.740218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 400/828] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:07.745708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9431f5-0929-4338-8912-f74ba9d8177f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:07.785137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c4b3f-d272-4075-9496-65096ab47e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:07.786102 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c4b3f-d272-4075-9496-65096ab47e6b None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:07.786391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c4b3f-d272-4075-9496-65096ab47e6b None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:07.786665 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c4b3f-d272-4075-9496-65096ab47e6b None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:07.787457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c4b3f-d272-4075-9496-65096ab47e6b None admin] RBAC: Adding query filter params (type=network) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:07.787762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c4b3f-d272-4075-9496-65096ab47e6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:07.789682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c4b3f-d272-4075-9496-65096ab47e6b None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:07.799433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 429/829] 10.222.0.25 () {56 vars in 1112 bytes} [Mon May 9 18:17:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:07.801435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c4b3f-d272-4075-9496-65096ab47e6b None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:07.902562 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9431f5-0929-4338-8912-f74ba9d8177f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:07.937228 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d42aa5-3b06-4c36-88ee-28ad7969a8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:07.937952 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d42aa5-3b06-4c36-88ee-28ad7969a8bf None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:07.938237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d42aa5-3b06-4c36-88ee-28ad7969a8bf None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:07.938548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d42aa5-3b06-4c36-88ee-28ad7969a8bf None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:07.939367 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d42aa5-3b06-4c36-88ee-28ad7969a8bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:07.939682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d42aa5-3b06-4c36-88ee-28ad7969a8bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:07.941107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d42aa5-3b06-4c36-88ee-28ad7969a8bf None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:07.962213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 401/830] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:17:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:07.971596 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d42aa5-3b06-4c36-88ee-28ad7969a8bf None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:07.982019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c4b3f-d272-4075-9496-65096ab47e6b None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:08.015330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dedd1-8d5e-409e-ba00-eb5adfedfbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:08.015990 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dedd1-8d5e-409e-ba00-eb5adfedfbd4 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:08.016270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dedd1-8d5e-409e-ba00-eb5adfedfbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:08.016602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dedd1-8d5e-409e-ba00-eb5adfedfbd4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:08.017302 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170dedd1-8d5e-409e-ba00-eb5adfedfbd4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:08.018645 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170dedd1-8d5e-409e-ba00-eb5adfedfbd4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:08.058075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 430/831] 10.222.0.25 () {60 vars in 1144 bytes} [Mon May 9 18:17:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:17:08.060611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f02e67-9768-41e6-8dbb-4d755c450d08 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:08.060878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f02e67-9768-41e6-8dbb-4d755c450d08 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:08.061155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f02e67-9768-41e6-8dbb-4d755c450d08 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:08.062861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 431/832] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:08.068660 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de4253-6926-44f2-bf6c-e99868dcfe6a None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:08.068972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de4253-6926-44f2-bf6c-e99868dcfe6a None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:08.069206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de4253-6926-44f2-bf6c-e99868dcfe6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:08.094244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b4de4253-6926-44f2-bf6c-e99868dcfe6a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:08.096114 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b4de4253-6926-44f2-bf6c-e99868dcfe6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:08.174465 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d42aa5-3b06-4c36-88ee-28ad7969a8bf None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:08.175114 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 432/833] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:08.198317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de4253-6926-44f2-bf6c-e99868dcfe6a None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:08.247387 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8d4c2-d994-48ed-97ec-4ee713f1f183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:08.248064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8d4c2-d994-48ed-97ec-4ee713f1f183 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:08.248343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8d4c2-d994-48ed-97ec-4ee713f1f183 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:08.248614 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8d4c2-d994-48ed-97ec-4ee713f1f183 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:08.249453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8d4c2-d994-48ed-97ec-4ee713f1f183 None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=f769ee9f31354e5296645fb1e3bd84b6) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:08.249759 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8d4c2-d994-48ed-97ec-4ee713f1f183 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:08.251244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8d4c2-d994-48ed-97ec-4ee713f1f183 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:08.272974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 402/834] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:17:07 2022] GET /identity/v3/role_assignments?user.id=f769ee9f31354e5296645fb1e3bd84b6&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:08.348350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de4253-6926-44f2-bf6c-e99868dcfe6a None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:08.390597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf0426-0f7c-4b49-8c4f-69dfacf7265c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:08.391985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf0426-0f7c-4b49-8c4f-69dfacf7265c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:08.392284 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf0426-0f7c-4b49-8c4f-69dfacf7265c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:08.392558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf0426-0f7c-4b49-8c4f-69dfacf7265c None admin] PATH_INFO: `/v3/roles/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:08.404728 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf0426-0f7c-4b49-8c4f-69dfacf7265c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-69bf0426-0f7c-4b49-8c4f-69dfacf7265c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:08.413420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:08.415500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:08.416519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:08.416519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:08.416519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:08.416519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:08.416519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:08.416519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:08.416519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:08.416519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:08.416519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:08.416519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:08.416519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:17:08.416519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:08.416519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 433/835] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:17:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:08.420487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8d4c2-d994-48ed-97ec-4ee713f1f183 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:08.533765 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8d4c2-d994-48ed-97ec-4ee713f1f183 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:08.569595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b26c0-9793-47a1-ab27-c9fcc95bd155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:08.570691 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b26c0-9793-47a1-ab27-c9fcc95bd155 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:08.571215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b26c0-9793-47a1-ab27-c9fcc95bd155 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:08.571714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b26c0-9793-47a1-ab27-c9fcc95bd155 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:08.572693 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b26c0-9793-47a1-ab27-c9fcc95bd155 None admin] RBAC: Adding query filter params (name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:08.573222 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b26c0-9793-47a1-ab27-c9fcc95bd155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:08.574826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b26c0-9793-47a1-ab27-c9fcc95bd155 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:08.575256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824491e9-4f63-4524-a453-9c8c4f5c4b9b None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:08.575687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824491e9-4f63-4524-a453-9c8c4f5c4b9b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:08.575846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824491e9-4f63-4524-a453-9c8c4f5c4b9b None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:08.578005 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 434/836] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:08.584171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a7e34-4710-44d2-b6e2-e0f01f417226 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:08.584408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 403/837] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:17:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:08.584521 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a7e34-4710-44d2-b6e2-e0f01f417226 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:08.584785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a7e34-4710-44d2-b6e2-e0f01f417226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:08.589739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b26c0-9793-47a1-ab27-c9fcc95bd155 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:08.607596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9b9a7e34-4710-44d2-b6e2-e0f01f417226 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:08.608762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9b9a7e34-4710-44d2-b6e2-e0f01f417226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:08.686424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 435/838] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:08.710876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a7e34-4710-44d2-b6e2-e0f01f417226 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:08.720064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b26c0-9793-47a1-ab27-c9fcc95bd155 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:08.753702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ac844-c2a6-4417-9368-29e9d7596e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:08.754494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ac844-c2a6-4417-9368-29e9d7596e88 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:08.754771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ac844-c2a6-4417-9368-29e9d7596e88 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:08.755040 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ac844-c2a6-4417-9368-29e9d7596e88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:08.765519 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ac844-c2a6-4417-9368-29e9d7596e88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:08.767924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ac844-c2a6-4417-9368-29e9d7596e88 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-a34ac844-c2a6-4417-9368-29e9d7596e88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:08.785542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:08.786445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:08.787376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:08.787376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:08.787376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:08.787376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:08.787376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 404/839] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:08.792400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ac844-c2a6-4417-9368-29e9d7596e88 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:08.809925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a7e34-4710-44d2-b6e2-e0f01f417226 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:08.844471 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35616dc6-fe38-4a97-b54d-f2d17e016204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:08.845404 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35616dc6-fe38-4a97-b54d-f2d17e016204 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:08.845686 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35616dc6-fe38-4a97-b54d-f2d17e016204 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:08.845958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35616dc6-fe38-4a97-b54d-f2d17e016204 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:08.856184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35616dc6-fe38-4a97-b54d-f2d17e016204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-35616dc6-fe38-4a97-b54d-f2d17e016204 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:08.864281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:08.866389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:08.868409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:08.868409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:08.868409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:08.868409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:08.868409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:08.868409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:08.868409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:08.868409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:08.868409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:08.868409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:08.868409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:17:08.868409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:08.875198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 436/840] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:17:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:08.875198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35616dc6-fe38-4a97-b54d-f2d17e016204 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:08.891628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ac844-c2a6-4417-9368-29e9d7596e88 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:08.924777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6b39d-fc99-4459-8e25-d6b7564f5237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:08.925447 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6b39d-fc99-4459-8e25-d6b7564f5237 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:08.925728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6b39d-fc99-4459-8e25-d6b7564f5237 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:08.926006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6b39d-fc99-4459-8e25-d6b7564f5237 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:08.926717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6b39d-fc99-4459-8e25-d6b7564f5237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:08.927020 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6b39d-fc99-4459-8e25-d6b7564f5237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:08.928394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6b39d-fc99-4459-8e25-d6b7564f5237 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:08.945904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 405/841] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:08.946892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6b39d-fc99-4459-8e25-d6b7564f5237 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:08.971065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35616dc6-fe38-4a97-b54d-f2d17e016204 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:09.011672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a6905-bed9-4a4d-b739-a51f11f03702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:09.013015 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a6905-bed9-4a4d-b739-a51f11f03702 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:09.013712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a6905-bed9-4a4d-b739-a51f11f03702 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:09.014128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a6905-bed9-4a4d-b739-a51f11f03702 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:09.015207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a6905-bed9-4a4d-b739-a51f11f03702 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:09.016033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a6905-bed9-4a4d-b739-a51f11f03702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:09.018002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a6905-bed9-4a4d-b739-a51f11f03702 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:09.028967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 437/842] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:17:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:09.034432 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a6905-bed9-4a4d-b739-a51f11f03702 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:09.062951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6b39d-fc99-4459-8e25-d6b7564f5237 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:09.097357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a5b41-a244-4615-9766-27b29c11e630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:09.098178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a5b41-a244-4615-9766-27b29c11e630 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:09.098779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a5b41-a244-4615-9766-27b29c11e630 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:09.098779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a5b41-a244-4615-9766-27b29c11e630 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:09.105852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a5b41-a244-4615-9766-27b29c11e630 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:09.107310 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a5b41-a244-4615-9766-27b29c11e630 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-753a5b41-a244-4615-9766-27b29c11e630 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:17:09.122390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:09.123608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:09.124612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:09.124612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:09.124612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:17:09.124612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:17:09.124612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:17:09.124612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:17:09.124612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:17:09.124612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:17:09.124612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 18:17:09.124612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:09.129182 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 406/843] 10.222.0.25 () {56 vars in 1117 bytes} [Mon May 9 18:17:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:09.130037 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a5b41-a244-4615-9766-27b29c11e630 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:09.157949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a6905-bed9-4a4d-b739-a51f11f03702 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:09.200805 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e6259-bd2f-4a3a-a35c-d72dce928868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:09.201894 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e6259-bd2f-4a3a-a35c-d72dce928868 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:09.202317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e6259-bd2f-4a3a-a35c-d72dce928868 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:09.202726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e6259-bd2f-4a3a-a35c-d72dce928868 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:09.217779 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e6259-bd2f-4a3a-a35c-d72dce928868 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:09.219249 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e6259-bd2f-4a3a-a35c-d72dce928868 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-2c6e6259-bd2f-4a3a-a35c-d72dce928868 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:09.237360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:09.238310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:09.239237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:09.239237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:09.239237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:09.239237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:09.242174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a5b41-a244-4615-9766-27b29c11e630 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:09.243278 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 438/844] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:09.244129 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e6259-bd2f-4a3a-a35c-d72dce928868 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:09.286085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bedd2a-94f5-4790-88eb-1221c56c4c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:09.287452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bedd2a-94f5-4790-88eb-1221c56c4c59 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:09.287941 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bedd2a-94f5-4790-88eb-1221c56c4c59 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:09.288643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bedd2a-94f5-4790-88eb-1221c56c4c59 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:09.289623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bedd2a-94f5-4790-88eb-1221c56c4c59 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:09.290232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bedd2a-94f5-4790-88eb-1221c56c4c59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:09.292442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bedd2a-94f5-4790-88eb-1221c56c4c59 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:09.316000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 407/845] 10.222.0.25 () {56 vars in 1149 bytes} [Mon May 9 18:17:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:09.321541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bedd2a-94f5-4790-88eb-1221c56c4c59 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:09.348899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e6259-bd2f-4a3a-a35c-d72dce928868 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:09.384418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b8722-ed3f-4f66-b199-88bd0fe45b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:09.385127 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b8722-ed3f-4f66-b199-88bd0fe45b35 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:09.385487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b8722-ed3f-4f66-b199-88bd0fe45b35 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:09.385908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b8722-ed3f-4f66-b199-88bd0fe45b35 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:09.386598 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b8722-ed3f-4f66-b199-88bd0fe45b35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:09.386976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b8722-ed3f-4f66-b199-88bd0fe45b35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:09.388428 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b8722-ed3f-4f66-b199-88bd0fe45b35 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:09.404855 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 439/846] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:09.405809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b8722-ed3f-4f66-b199-88bd0fe45b35 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:09.448483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bedd2a-94f5-4790-88eb-1221c56c4c59 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:09.490586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3600c-a37a-4cea-b692-79d36b741f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:09.491510 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3600c-a37a-4cea-b692-79d36b741f82 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:09.492386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3600c-a37a-4cea-b692-79d36b741f82 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:09.492837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3600c-a37a-4cea-b692-79d36b741f82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:09.505096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b8722-ed3f-4f66-b199-88bd0fe45b35 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:09.546014 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a0a91-d50c-4afe-ac2d-fe131180b0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:09.547581 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a0a91-d50c-4afe-ac2d-fe131180b0d3 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:09.548614 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a0a91-d50c-4afe-ac2d-fe131180b0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:09.549206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a0a91-d50c-4afe-ac2d-fe131180b0d3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:09.559362 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a0a91-d50c-4afe-ac2d-fe131180b0d3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:09.561488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a0a91-d50c-4afe-ac2d-fe131180b0d3 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:09.577394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f3600c-a37a-4cea-b692-79d36b741f82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:09.579443 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f3600c-a37a-4cea-b692-79d36b741f82 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-c73a0a91-d50c-4afe-ac2d-fe131180b0d3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:17:09.581328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:09.582767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:09.583831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:09.583831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:09.583831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:17:09.583831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:17:09.583831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:17:09.583831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:17:09.583831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:17:09.583831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:17:09.583831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 18:17:09.583831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:09.587501 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 440/847] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:17:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:09.590761 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a0a91-d50c-4afe-ac2d-fe131180b0d3 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:09.687368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 408/848] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:17:09 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 18:17:09.692628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3600c-a37a-4cea-b692-79d36b741f82 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:09.701498 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a0a91-d50c-4afe-ac2d-fe131180b0d3 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:09.738018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c289482-ae8f-4761-9b69-2608d24094e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:09.738691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c289482-ae8f-4761-9b69-2608d24094e5 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:09.738975 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c289482-ae8f-4761-9b69-2608d24094e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:09.739553 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c289482-ae8f-4761-9b69-2608d24094e5 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:09.740104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c289482-ae8f-4761-9b69-2608d24094e5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:09.740410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c289482-ae8f-4761-9b69-2608d24094e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:09.741817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c289482-ae8f-4761-9b69-2608d24094e5 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:09.763020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 441/849] 10.222.0.25 () {56 vars in 1135 bytes} [Mon May 9 18:17:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:09.763020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32249612-dadc-4ae3-9bdc-80cbecd6f3b5 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:09.763020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32249612-dadc-4ae3-9bdc-80cbecd6f3b5 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:09.763020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32249612-dadc-4ae3-9bdc-80cbecd6f3b5 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:09.763020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 442/850] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:09.763778 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32249612-dadc-4ae3-9bdc-80cbecd6f3b5 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:09.797666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3600c-a37a-4cea-b692-79d36b741f82 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:09.829792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e284017-73e3-4253-a31a-298fc757ef10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:09.830511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e284017-73e3-4253-a31a-298fc757ef10 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:09.830785 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e284017-73e3-4253-a31a-298fc757ef10 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:09.831053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e284017-73e3-4253-a31a-298fc757ef10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:09.840942 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e284017-73e3-4253-a31a-298fc757ef10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:09.842357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e284017-73e3-4253-a31a-298fc757ef10 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-6e284017-73e3-4253-a31a-298fc757ef10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:09.860576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:09.861503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:09.862480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:09.862480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:09.862480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:09.862480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:09.862480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 409/851] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:09.863309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e2b4a-f6bf-4e43-b32c-71f01794c623 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:09.863309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e2b4a-f6bf-4e43-b32c-71f01794c623 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:09.863525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e2b4a-f6bf-4e43-b32c-71f01794c623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:09.876716 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32249612-dadc-4ae3-9bdc-80cbecd6f3b5 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:09.879249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f36e2b4a-f6bf-4e43-b32c-71f01794c623 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:09.880395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f36e2b4a-f6bf-4e43-b32c-71f01794c623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:09.910886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0387b42-477b-45b0-a56b-dcd6cdb8f320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:09.911551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0387b42-477b-45b0-a56b-dcd6cdb8f320 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:09.911831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0387b42-477b-45b0-a56b-dcd6cdb8f320 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:09.912100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0387b42-477b-45b0-a56b-dcd6cdb8f320 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:09.948400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 410/852] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:09.949954 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e2b4a-f6bf-4e43-b32c-71f01794c623 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:09.984568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0387b42-477b-45b0-a56b-dcd6cdb8f320 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:09.986045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0387b42-477b-45b0-a56b-dcd6cdb8f320 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:10.058079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e2b4a-f6bf-4e43-b32c-71f01794c623 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:10.079936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 443/853] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:17:09 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 18:17:10.081094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0387b42-477b-45b0-a56b-dcd6cdb8f320 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:10.098084 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ac0ff-99d8-46ac-8287-a5ad3729edd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:10.099006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ac0ff-99d8-46ac-8287-a5ad3729edd3 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:10.099467 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ac0ff-99d8-46ac-8287-a5ad3729edd3 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:10.099890 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ac0ff-99d8-46ac-8287-a5ad3729edd3 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:10.100757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ac0ff-99d8-46ac-8287-a5ad3729edd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:10.101212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ac0ff-99d8-46ac-8287-a5ad3729edd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:10.102741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ac0ff-99d8-46ac-8287-a5ad3729edd3 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:10.114509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 411/854] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:10.115431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ac0ff-99d8-46ac-8287-a5ad3729edd3 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:10.180320 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0387b42-477b-45b0-a56b-dcd6cdb8f320 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:10.215054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa658438-9ef7-4511-b854-cbabc6c860b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:10.216189 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa658438-9ef7-4511-b854-cbabc6c860b8 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:10.216471 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa658438-9ef7-4511-b854-cbabc6c860b8 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:10.216747 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa658438-9ef7-4511-b854-cbabc6c860b8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:10.217263 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ac0ff-99d8-46ac-8287-a5ad3729edd3 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:10.217368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa658438-9ef7-4511-b854-cbabc6c860b8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:10.218725 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa658438-9ef7-4511-b854-cbabc6c860b8 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-aa658438-9ef7-4511-b854-cbabc6c860b8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:10.230221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:10.231257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:10.232092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:10.232092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:10.232092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:10.232092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:10.232092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:10.232092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:10.232092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:10.232092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:10.232092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:10.232092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:10.232092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:10.232092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:10.232092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 18:17:10.232092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:10.235328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 444/855] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:17:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:10.236430 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa658438-9ef7-4511-b854-cbabc6c860b8 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:10.251444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ecb47-b0ea-485a-9b3f-ad22c2ef2e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:10.252591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ecb47-b0ea-485a-9b3f-ad22c2ef2e1f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:10.252928 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ecb47-b0ea-485a-9b3f-ad22c2ef2e1f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:10.253208 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ecb47-b0ea-485a-9b3f-ad22c2ef2e1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:10.263794 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ecb47-b0ea-485a-9b3f-ad22c2ef2e1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:10.265235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ecb47-b0ea-485a-9b3f-ad22c2ef2e1f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-c07ecb47-b0ea-485a-9b3f-ad22c2ef2e1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:10.282631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:10.283742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:10.289124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:10.289124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:10.289124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:10.289124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:10.289124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 412/856] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:10.289124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ecb47-b0ea-485a-9b3f-ad22c2ef2e1f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:10.346405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa658438-9ef7-4511-b854-cbabc6c860b8 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:10.379282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e55217-c61f-4d77-bdb5-4a04d3bab09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:10.380414 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e55217-c61f-4d77-bdb5-4a04d3bab09e None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:10.380912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e55217-c61f-4d77-bdb5-4a04d3bab09e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:10.381288 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e55217-c61f-4d77-bdb5-4a04d3bab09e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:10.384469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ecb47-b0ea-485a-9b3f-ad22c2ef2e1f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:10.387812 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e55217-c61f-4d77-bdb5-4a04d3bab09e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:10.389293 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e55217-c61f-4d77-bdb5-4a04d3bab09e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-a5e55217-c61f-4d77-bdb5-4a04d3bab09e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:17:10.403281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:10.404207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:10.405101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:17:10.405101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:17:10.405101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:17:10.405101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:17:10.405101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:17:10.405101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:17:10.405101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:10.405101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:10.407662 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 445/857] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:10.408413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e55217-c61f-4d77-bdb5-4a04d3bab09e None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:10.417676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b3025-1b22-4028-9cb2-c618d5a7b6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:10.418348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b3025-1b22-4028-9cb2-c618d5a7b6ae None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:10.418738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b3025-1b22-4028-9cb2-c618d5a7b6ae None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:10.419016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b3025-1b22-4028-9cb2-c618d5a7b6ae None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:10.419753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b3025-1b22-4028-9cb2-c618d5a7b6ae None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:10.420057 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b3025-1b22-4028-9cb2-c618d5a7b6ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:10.421408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b3025-1b22-4028-9cb2-c618d5a7b6ae None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:10.431349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 413/858] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:17:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:10.432288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b3025-1b22-4028-9cb2-c618d5a7b6ae None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:10.508375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e55217-c61f-4d77-bdb5-4a04d3bab09e None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:10.538242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b3025-1b22-4028-9cb2-c618d5a7b6ae None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:10.545315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a122c4-4749-466f-8d85-65ec7d0d2f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:10.546013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a122c4-4749-466f-8d85-65ec7d0d2f44 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:10.546326 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a122c4-4749-466f-8d85-65ec7d0d2f44 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:10.546648 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a122c4-4749-466f-8d85-65ec7d0d2f44 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:10.547516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a122c4-4749-466f-8d85-65ec7d0d2f44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:10.548342 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a122c4-4749-466f-8d85-65ec7d0d2f44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:10.549744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a122c4-4749-466f-8d85-65ec7d0d2f44 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:10.561370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 446/859] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:10.562185 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a122c4-4749-466f-8d85-65ec7d0d2f44 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:10.577329 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc751d3-2b94-4f06-915e-6582f03667f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:10.578125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc751d3-2b94-4f06-915e-6582f03667f8 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:10.578411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc751d3-2b94-4f06-915e-6582f03667f8 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:10.578688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc751d3-2b94-4f06-915e-6582f03667f8 None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:10.579520 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc751d3-2b94-4f06-915e-6582f03667f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:10.579828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc751d3-2b94-4f06-915e-6582f03667f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:10.581226 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc751d3-2b94-4f06-915e-6582f03667f8 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:10.592929 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 414/860] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:17:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:10.593530 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc751d3-2b94-4f06-915e-6582f03667f8 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:10.671845 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a122c4-4749-466f-8d85-65ec7d0d2f44 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:10.694146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc751d3-2b94-4f06-915e-6582f03667f8 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:10.704567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e34e58-d982-4e10-97bc-9a39a8bd8b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:10.705231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e34e58-d982-4e10-97bc-9a39a8bd8b7f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:10.705512 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e34e58-d982-4e10-97bc-9a39a8bd8b7f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:10.705786 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e34e58-d982-4e10-97bc-9a39a8bd8b7f None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:10.706481 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e34e58-d982-4e10-97bc-9a39a8bd8b7f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:10.706786 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e34e58-d982-4e10-97bc-9a39a8bd8b7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:10.708263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e34e58-d982-4e10-97bc-9a39a8bd8b7f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:10.712493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 447/861] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:17:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:10.713267 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e34e58-d982-4e10-97bc-9a39a8bd8b7f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:10.728059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12db0d-4055-4257-90b8-29fd65460f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:10.729152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12db0d-4055-4257-90b8-29fd65460f24 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:10.729432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12db0d-4055-4257-90b8-29fd65460f24 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:10.729703 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12db0d-4055-4257-90b8-29fd65460f24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:10.737771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12db0d-4055-4257-90b8-29fd65460f24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:10.739260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12db0d-4055-4257-90b8-29fd65460f24 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-df12db0d-4055-4257-90b8-29fd65460f24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:17:10.754050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:10.755247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:10.756351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:17:10.756351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:17:10.756351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:17:10.756351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:17:10.756351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:17:10.756351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:17:10.756351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:10.756351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:10.756351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 415/862] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:10.756944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5011fac-48d0-4a8a-91c9-221a401e7652 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:10.756944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5011fac-48d0-4a8a-91c9-221a401e7652 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:10.757164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5011fac-48d0-4a8a-91c9-221a401e7652 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:10.762623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 416/863] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:10.763574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5011fac-48d0-4a8a-91c9-221a401e7652 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:10.829667 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e34e58-d982-4e10-97bc-9a39a8bd8b7f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:10.865539 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4547e-d879-4b81-8d0b-5b34020131c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:10.866206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4547e-d879-4b81-8d0b-5b34020131c5 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:10.866511 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4547e-d879-4b81-8d0b-5b34020131c5 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:10.866787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4547e-d879-4b81-8d0b-5b34020131c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:10.867651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4547e-d879-4b81-8d0b-5b34020131c5 None admin] RBAC: Adding query filter params (role.id=7b2e521b45dc4a98adad996ca0027de6, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=154f86b7575e47efb2a5e1c35943e037) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:10.867959 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4547e-d879-4b81-8d0b-5b34020131c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:10.869268 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5011fac-48d0-4a8a-91c9-221a401e7652 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:10.869399 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4547e-d879-4b81-8d0b-5b34020131c5 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:10.878820 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 448/864] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:17:10 2022] GET /identity/v3/role_assignments?user.id=154f86b7575e47efb2a5e1c35943e037&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=7b2e521b45dc4a98adad996ca0027de6 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:10.880436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c02b0e-db04-4544-ae4c-8655093dc940 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:10.880720 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c02b0e-db04-4544-ae4c-8655093dc940 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:10.880996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c02b0e-db04-4544-ae4c-8655093dc940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:10.899920 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-42c02b0e-db04-4544-ae4c-8655093dc940 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:10.901290 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-42c02b0e-db04-4544-ae4c-8655093dc940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:10.919610 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60214b-e7ca-40ce-8dd4-91728aa9ded5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:10.920431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60214b-e7ca-40ce-8dd4-91728aa9ded5 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:10.920612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60214b-e7ca-40ce-8dd4-91728aa9ded5 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:10.920885 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60214b-e7ca-40ce-8dd4-91728aa9ded5 None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:10.921679 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60214b-e7ca-40ce-8dd4-91728aa9ded5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:10.922038 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60214b-e7ca-40ce-8dd4-91728aa9ded5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:10.924109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60214b-e7ca-40ce-8dd4-91728aa9ded5 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:10.943206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 417/865] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:17:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:10.946750 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60214b-e7ca-40ce-8dd4-91728aa9ded5 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:10.994971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 449/866] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:11.017835 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c02b0e-db04-4544-ae4c-8655093dc940 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:11.063607 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60214b-e7ca-40ce-8dd4-91728aa9ded5 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:11.104377 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e152d-ffc7-4eb2-86c5-8f5d33452e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:11.105045 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e152d-ffc7-4eb2-86c5-8f5d33452e59 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:11.105323 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e152d-ffc7-4eb2-86c5-8f5d33452e59 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:11.105595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e152d-ffc7-4eb2-86c5-8f5d33452e59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:11.106418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e152d-ffc7-4eb2-86c5-8f5d33452e59 None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=ec2db3aa2b7d41f89314fec960d898b7) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:11.106728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e152d-ffc7-4eb2-86c5-8f5d33452e59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:11.108307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e152d-ffc7-4eb2-86c5-8f5d33452e59 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:11.117215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 418/867] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:17:10 2022] GET /identity/v3/role_assignments?user.id=ec2db3aa2b7d41f89314fec960d898b7&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:11.126490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c02b0e-db04-4544-ae4c-8655093dc940 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:11.166990 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c6f20-25e9-4a4f-b1e6-f0468e79c3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:11.167915 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c6f20-25e9-4a4f-b1e6-f0468e79c3cd None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:11.168200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c6f20-25e9-4a4f-b1e6-f0468e79c3cd None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:11.168476 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c6f20-25e9-4a4f-b1e6-f0468e79c3cd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:11.169184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c6f20-25e9-4a4f-b1e6-f0468e79c3cd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:11.170584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c6f20-25e9-4a4f-b1e6-f0468e79c3cd None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-623c6f20-25e9-4a4f-b1e6-f0468e79c3cd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:11.184107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:11.185097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:11.185962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 450/868] 10.222.0.25 () {56 vars in 1132 bytes} [Mon May 9 18:17:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:11.191495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e152d-ffc7-4eb2-86c5-8f5d33452e59 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:11.308495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e152d-ffc7-4eb2-86c5-8f5d33452e59 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:11.342858 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62870d2b-4e12-4da0-ab9a-4f8da3fc016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:11.343571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62870d2b-4e12-4da0-ab9a-4f8da3fc016a None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:11.343855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62870d2b-4e12-4da0-ab9a-4f8da3fc016a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:11.344134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62870d2b-4e12-4da0-ab9a-4f8da3fc016a None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:11.345412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62870d2b-4e12-4da0-ab9a-4f8da3fc016a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:11.345720 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62870d2b-4e12-4da0-ab9a-4f8da3fc016a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:11.347090 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62870d2b-4e12-4da0-ab9a-4f8da3fc016a None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:11.352213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 419/869] 10.222.0.25 () {56 vars in 1126 bytes} [Mon May 9 18:17:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:11.360053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c6f20-25e9-4a4f-b1e6-f0468e79c3cd None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:11.460311 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c6f20-25e9-4a4f-b1e6-f0468e79c3cd None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:11.493631 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c688889-f73e-4a37-af2a-dacb76408d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:11.494286 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c688889-f73e-4a37-af2a-dacb76408d9e None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:11.494564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c688889-f73e-4a37-af2a-dacb76408d9e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:11.494835 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c688889-f73e-4a37-af2a-dacb76408d9e None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:11.495553 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c688889-f73e-4a37-af2a-dacb76408d9e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:11.495849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c688889-f73e-4a37-af2a-dacb76408d9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:11.497190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c688889-f73e-4a37-af2a-dacb76408d9e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:11.502671 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 451/870] 10.222.0.25 () {56 vars in 1126 bytes} [Mon May 9 18:17:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:13.016917 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c82fd-0c07-404c-bafe-f0f46935bb21 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:13.017897 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c82fd-0c07-404c-bafe-f0f46935bb21 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:13.018379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c82fd-0c07-404c-bafe-f0f46935bb21 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:13.020263 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 420/871] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:13.025140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab98e1e-d3ac-4259-8b49-efb45718b991 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:13.025563 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab98e1e-d3ac-4259-8b49-efb45718b991 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:13.025758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab98e1e-d3ac-4259-8b49-efb45718b991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:13.042846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-8ab98e1e-d3ac-4259-8b49-efb45718b991 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:13.044036 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-8ab98e1e-d3ac-4259-8b49-efb45718b991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:13.116101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 452/872] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:13.138063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c82fd-0c07-404c-bafe-f0f46935bb21 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:13.241996 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c82fd-0c07-404c-bafe-f0f46935bb21 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:13.261771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ea1fb-1342-40f2-91be-ee3419909395 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:13.262082 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ea1fb-1342-40f2-91be-ee3419909395 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:13.262336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ea1fb-1342-40f2-91be-ee3419909395 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:13.264048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 453/873] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:13.270131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f632ab7-49b9-4ac0-98a6-c8f1f45fee4a None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:13.270424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f632ab7-49b9-4ac0-98a6-c8f1f45fee4a None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:13.270697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f632ab7-49b9-4ac0-98a6-c8f1f45fee4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:13.280078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e1035-8bba-49d9-bf34-32dc8ce5b6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:13.280745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e1035-8bba-49d9-bf34-32dc8ce5b6d8 None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:13.281024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e1035-8bba-49d9-bf34-32dc8ce5b6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:13.281290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e1035-8bba-49d9-bf34-32dc8ce5b6d8 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:13.282540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e1035-8bba-49d9-bf34-32dc8ce5b6d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:13.283911 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e1035-8bba-49d9-bf34-32dc8ce5b6d8 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:13.289382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-8f632ab7-49b9-4ac0-98a6-c8f1f45fee4a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:13.290704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-8f632ab7-49b9-4ac0-98a6-c8f1f45fee4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:13.299239 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 421/874] 10.222.0.25 () {60 vars in 1141 bytes} [Mon May 9 18:17:13 2022] POST /identity/v3/services => generated 251 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:17:13.374204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 454/875] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:13.399296 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e1035-8bba-49d9-bf34-32dc8ce5b6d8 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:13.445031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16fbf9-2e11-4c72-8bd8-3cedd707a978 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:13.445326 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16fbf9-2e11-4c72-8bd8-3cedd707a978 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:13.445678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16fbf9-2e11-4c72-8bd8-3cedd707a978 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:13.451366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 455/876] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:13.453165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe046fe-32bf-47ca-b01d-9855c6373896 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:13.453554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe046fe-32bf-47ca-b01d-9855c6373896 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:13.454438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe046fe-32bf-47ca-b01d-9855c6373896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:13.472240 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-1fe046fe-32bf-47ca-b01d-9855c6373896 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:13.473563 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-1fe046fe-32bf-47ca-b01d-9855c6373896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:13.517492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e1035-8bba-49d9-bf34-32dc8ce5b6d8 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:13.554718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807de673-0db6-4f08-a4d5-029e4127f884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:13.555617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807de673-0db6-4f08-a4d5-029e4127f884 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:13.555902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807de673-0db6-4f08-a4d5-029e4127f884 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:13.556182 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807de673-0db6-4f08-a4d5-029e4127f884 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:13.563419 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 456/877] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:13.569875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807de673-0db6-4f08-a4d5-029e4127f884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-807de673-0db6-4f08-a4d5-029e4127f884 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:13.578379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:13.579946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:13.581232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:13.581232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:13.581232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:13.581232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:13.581232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:13.581232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:13.581232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:13.581232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:13.581232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:13.581232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:13.581232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 18:17:13.581232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:13.581232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 422/878] 10.222.0.25 () {56 vars in 1120 bytes} [Mon May 9 18:17:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:13.586474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe046fe-32bf-47ca-b01d-9855c6373896 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:13.591106 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807de673-0db6-4f08-a4d5-029e4127f884 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:13.685460 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe046fe-32bf-47ca-b01d-9855c6373896 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:13.691103 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807de673-0db6-4f08-a4d5-029e4127f884 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:13.719115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12715a09-d3a1-4d2a-b3f3-8121a403abf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:13.720524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12715a09-d3a1-4d2a-b3f3-8121a403abf0 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:13.720747 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12715a09-d3a1-4d2a-b3f3-8121a403abf0 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:13.721025 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12715a09-d3a1-4d2a-b3f3-8121a403abf0 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:13.722016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12715a09-d3a1-4d2a-b3f3-8121a403abf0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:13.722278 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12715a09-d3a1-4d2a-b3f3-8121a403abf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:13.723789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12715a09-d3a1-4d2a-b3f3-8121a403abf0 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:13.727352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b336e-3c35-4d6e-9926-f46dd40b65a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:13.728402 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b336e-3c35-4d6e-9926-f46dd40b65a0 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:13.728902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b336e-3c35-4d6e-9926-f46dd40b65a0 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:13.729392 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b336e-3c35-4d6e-9926-f46dd40b65a0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:13.730247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b336e-3c35-4d6e-9926-f46dd40b65a0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:13.731831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b336e-3c35-4d6e-9926-f46dd40b65a0 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:13.736816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 457/879] 10.222.0.25 () {56 vars in 1115 bytes} [Mon May 9 18:17:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:13.737743 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12715a09-d3a1-4d2a-b3f3-8121a403abf0 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-0e3b336e-3c35-4d6e-9926-f46dd40b65a0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:13.745615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:13.746629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:13.747493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:13.747493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:13.747493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:13.747493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:13.747493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:13.747493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:13.747493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:13.747493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:13.747493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:13.747493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:13.747493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:13.747493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:13.747493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:13.747493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:13.748821 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 423/880] 10.222.0.25 () {56 vars in 1126 bytes} [Mon May 9 18:17:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:13.750316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61cbbf-7434-4d45-9206-b9a2e8ff87ec None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:13.751376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61cbbf-7434-4d45-9206-b9a2e8ff87ec None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:13.751659 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61cbbf-7434-4d45-9206-b9a2e8ff87ec None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:13.753206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 424/881] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:13.754988 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61cbbf-7434-4d45-9206-b9a2e8ff87ec None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:13.836525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12715a09-d3a1-4d2a-b3f3-8121a403abf0 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:13.858317 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61cbbf-7434-4d45-9206-b9a2e8ff87ec None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:13.868312 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e4503-a67f-41a0-8ec3-c25ff026a09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:13.869103 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e4503-a67f-41a0-8ec3-c25ff026a09f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:13.869380 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e4503-a67f-41a0-8ec3-c25ff026a09f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:13.869651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e4503-a67f-41a0-8ec3-c25ff026a09f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:13.883090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e4503-a67f-41a0-8ec3-c25ff026a09f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:13.884554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e4503-a67f-41a0-8ec3-c25ff026a09f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:13.892972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec73488-68c8-46c6-bb16-e0f993534092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:13.893635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec73488-68c8-46c6-bb16-e0f993534092 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:13.893911 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec73488-68c8-46c6-bb16-e0f993534092 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:13.894183 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec73488-68c8-46c6-bb16-e0f993534092 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:13.894876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec73488-68c8-46c6-bb16-e0f993534092 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:13.896845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec73488-68c8-46c6-bb16-e0f993534092 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:13.896845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec73488-68c8-46c6-bb16-e0f993534092 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:13.905489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 425/882] 10.222.0.25 () {56 vars in 1122 bytes} [Mon May 9 18:17:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:13.905489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df528671-bc7f-47c2-a5d1-410894529c8f None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:13.905489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df528671-bc7f-47c2-a5d1-410894529c8f None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-2c7e4503-a67f-41a0-8ec3-c25ff026a09f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:13.905804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:13.906784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:13.908682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:13.908682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:13.908682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:13.908682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:13.908949 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df528671-bc7f-47c2-a5d1-410894529c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:13.909051 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 458/883] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:13.909302 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e4503-a67f-41a0-8ec3-c25ff026a09f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:13.922844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-df528671-bc7f-47c2-a5d1-410894529c8f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:13.924051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-df528671-bc7f-47c2-a5d1-410894529c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:13.989755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 426/884] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:13.990422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df528671-bc7f-47c2-a5d1-410894529c8f None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:14.021998 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e4503-a67f-41a0-8ec3-c25ff026a09f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:14.053930 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e37e0-7cf8-4045-809c-7dd520522e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:14.054687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e37e0-7cf8-4045-809c-7dd520522e8c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:14.054899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e37e0-7cf8-4045-809c-7dd520522e8c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:14.055190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e37e0-7cf8-4045-809c-7dd520522e8c None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:14.055911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e37e0-7cf8-4045-809c-7dd520522e8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:14.056250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e37e0-7cf8-4045-809c-7dd520522e8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:14.057601 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e37e0-7cf8-4045-809c-7dd520522e8c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:14.074150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 459/885] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:14.074150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e37e0-7cf8-4045-809c-7dd520522e8c None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:14.087044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df528671-bc7f-47c2-a5d1-410894529c8f None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:14.120105 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c28df3-7270-42bd-9362-f09b7ba480fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:14.120772 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c28df3-7270-42bd-9362-f09b7ba480fb None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:14.121056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c28df3-7270-42bd-9362-f09b7ba480fb None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:14.121333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c28df3-7270-42bd-9362-f09b7ba480fb None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:14.122034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c28df3-7270-42bd-9362-f09b7ba480fb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:14.122341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c28df3-7270-42bd-9362-f09b7ba480fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:14.123733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c28df3-7270-42bd-9362-f09b7ba480fb None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:14.135321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 427/886] 10.222.0.25 () {56 vars in 1122 bytes} [Mon May 9 18:17:13 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:14.135321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c28df3-7270-42bd-9362-f09b7ba480fb None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:14.177111 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e37e0-7cf8-4045-809c-7dd520522e8c None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:14.213746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963fc08-abfd-4040-8f28-791297082153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:14.214465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963fc08-abfd-4040-8f28-791297082153 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:14.214749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963fc08-abfd-4040-8f28-791297082153 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:14.215019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963fc08-abfd-4040-8f28-791297082153 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:14.215764 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963fc08-abfd-4040-8f28-791297082153 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:14.217162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963fc08-abfd-4040-8f28-791297082153 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:14.236127 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 460/887] 10.222.0.25 () {60 vars in 1141 bytes} [Mon May 9 18:17:14 2022] POST /identity/v3/services => generated 269 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:17:14.252883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c28df3-7270-42bd-9362-f09b7ba480fb None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:14.300492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a4b5c-c140-49dd-8bec-6d3d21af8594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:14.301474 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a4b5c-c140-49dd-8bec-6d3d21af8594 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:14.301702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a4b5c-c140-49dd-8bec-6d3d21af8594 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:14.301984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a4b5c-c140-49dd-8bec-6d3d21af8594 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:14.310904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a4b5c-c140-49dd-8bec-6d3d21af8594 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:14.312484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a4b5c-c140-49dd-8bec-6d3d21af8594 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-b09a4b5c-c140-49dd-8bec-6d3d21af8594 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:17:14.330816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:14.331883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:14.332832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:14.332832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:14.332832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:17:14.332832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:17:14.332832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:17:14.332832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:17:14.332832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:17:14.332832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:17:14.332832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 18:17:14.332832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:14.337187 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 428/888] 10.222.0.25 () {56 vars in 1117 bytes} [Mon May 9 18:17:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:14.340280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963fc08-abfd-4040-8f28-791297082153 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:14.446817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963fc08-abfd-4040-8f28-791297082153 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:14.481125 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acb8f0-280f-4027-bd4b-71aac324b54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:14.482347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acb8f0-280f-4027-bd4b-71aac324b54f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:14.482833 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acb8f0-280f-4027-bd4b-71aac324b54f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:14.483317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acb8f0-280f-4027-bd4b-71aac324b54f None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:14.484589 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acb8f0-280f-4027-bd4b-71aac324b54f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:14.485077 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acb8f0-280f-4027-bd4b-71aac324b54f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:14.487169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acb8f0-280f-4027-bd4b-71aac324b54f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:14.506475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 461/889] 10.222.0.25 () {56 vars in 1149 bytes} [Mon May 9 18:17:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:14.514592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a4b5c-c140-49dd-8bec-6d3d21af8594 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:14.620374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a4b5c-c140-49dd-8bec-6d3d21af8594 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:14.654729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c0395-65b8-4a0a-8ca7-6d9ec443af39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:14.655432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c0395-65b8-4a0a-8ca7-6d9ec443af39 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:14.655717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c0395-65b8-4a0a-8ca7-6d9ec443af39 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:14.655996 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c0395-65b8-4a0a-8ca7-6d9ec443af39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:14.726659 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c0395-65b8-4a0a-8ca7-6d9ec443af39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:14.728129 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c0395-65b8-4a0a-8ca7-6d9ec443af39 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:14.828630 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 429/890] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:17:14 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:17:14.830544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acb8f0-280f-4027-bd4b-71aac324b54f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:14.928366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acb8f0-280f-4027-bd4b-71aac324b54f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:14.966790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42430f-e8f2-49d6-9e22-e5cb436ee7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:14.967891 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42430f-e8f2-49d6-9e22-e5cb436ee7f9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:14.968405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42430f-e8f2-49d6-9e22-e5cb436ee7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:14.968897 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42430f-e8f2-49d6-9e22-e5cb436ee7f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:14.980188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42430f-e8f2-49d6-9e22-e5cb436ee7f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:14.981940 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42430f-e8f2-49d6-9e22-e5cb436ee7f9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-8c42430f-e8f2-49d6-9e22-e5cb436ee7f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:14.999830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:15.000861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:15.001785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:15.001785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:15.001785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:15.001785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:15.008570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 462/891] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:15.010178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c0395-65b8-4a0a-8ca7-6d9ec443af39 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:15.111272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c0395-65b8-4a0a-8ca7-6d9ec443af39 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:15.146602 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c2b01-f34f-4860-8b4d-6487211f8899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:15.147733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c2b01-f34f-4860-8b4d-6487211f8899 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:15.148221 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c2b01-f34f-4860-8b4d-6487211f8899 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:15.148698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c2b01-f34f-4860-8b4d-6487211f8899 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:15.149619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c2b01-f34f-4860-8b4d-6487211f8899 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:15.150125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c2b01-f34f-4860-8b4d-6487211f8899 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:15.151867 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c2b01-f34f-4860-8b4d-6487211f8899 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:15.168925 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 430/892] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:15.170549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42430f-e8f2-49d6-9e22-e5cb436ee7f9 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:15.270556 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42430f-e8f2-49d6-9e22-e5cb436ee7f9 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:15.304441 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bae4b-55ed-46d5-a2f1-ccc4acb3b0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:15.305599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bae4b-55ed-46d5-a2f1-ccc4acb3b0ff None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:15.306096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bae4b-55ed-46d5-a2f1-ccc4acb3b0ff None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:15.306576 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bae4b-55ed-46d5-a2f1-ccc4acb3b0ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:15.314019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bae4b-55ed-46d5-a2f1-ccc4acb3b0ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:15.315703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bae4b-55ed-46d5-a2f1-ccc4acb3b0ff None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-474bae4b-55ed-46d5-a2f1-ccc4acb3b0ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:17:15.330105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:15.330987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:15.331949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:17:15.331949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:17:15.331949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:17:15.331949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:17:15.331949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:17:15.331949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:17:15.331949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:15.331949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:15.338234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 463/893] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:15.340460 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c2b01-f34f-4860-8b4d-6487211f8899 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:15.446598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c2b01-f34f-4860-8b4d-6487211f8899 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:15.485908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d30d44-18c8-4ca0-b094-e8589658ee2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:15.486960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d30d44-18c8-4ca0-b094-e8589658ee2b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:15.487502 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d30d44-18c8-4ca0-b094-e8589658ee2b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:15.487984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d30d44-18c8-4ca0-b094-e8589658ee2b None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:15.488976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d30d44-18c8-4ca0-b094-e8589658ee2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:15.489489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d30d44-18c8-4ca0-b094-e8589658ee2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:15.491122 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d30d44-18c8-4ca0-b094-e8589658ee2b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:15.510602 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 431/894] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:17:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:15.512355 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bae4b-55ed-46d5-a2f1-ccc4acb3b0ff None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:15.613923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bae4b-55ed-46d5-a2f1-ccc4acb3b0ff None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:15.647869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00b818-d14f-458d-8879-069dc5b8a3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:15.648863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00b818-d14f-458d-8879-069dc5b8a3f3 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:15.649366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00b818-d14f-458d-8879-069dc5b8a3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:15.649836 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00b818-d14f-458d-8879-069dc5b8a3f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:15.650862 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00b818-d14f-458d-8879-069dc5b8a3f3 None admin] RBAC: Adding query filter params (role.id=104650f78a264c78a8cf5c6c4b389575, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=154f86b7575e47efb2a5e1c35943e037) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:15.651395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00b818-d14f-458d-8879-069dc5b8a3f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:15.653069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00b818-d14f-458d-8879-069dc5b8a3f3 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:15.655123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14da44-db10-4e6c-8b94-b5c62f012af9 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:15.655389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14da44-db10-4e6c-8b94-b5c62f012af9 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:15.655669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14da44-db10-4e6c-8b94-b5c62f012af9 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:15.657066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 432/895] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:17:15.664900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23401c6-3d08-4782-a498-0a13e0b41ed6 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:15.664900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23401c6-3d08-4782-a498-0a13e0b41ed6 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:15.664900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23401c6-3d08-4782-a498-0a13e0b41ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:15.665432 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 464/896] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:17:15 2022] GET /identity/v3/role_assignments?user.id=154f86b7575e47efb2a5e1c35943e037&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=104650f78a264c78a8cf5c6c4b389575 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:15.681449 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f23401c6-3d08-4782-a498-0a13e0b41ed6 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:15.682843 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f23401c6-3d08-4782-a498-0a13e0b41ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:15.753440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 433/897] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:15.775908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00b818-d14f-458d-8879-069dc5b8a3f3 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:15.882088 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00b818-d14f-458d-8879-069dc5b8a3f3 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:15.915359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524cf41-76bf-4082-8581-ac7b17eb6722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:15.916599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524cf41-76bf-4082-8581-ac7b17eb6722 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:15.917107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524cf41-76bf-4082-8581-ac7b17eb6722 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:15.917574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524cf41-76bf-4082-8581-ac7b17eb6722 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:15.918403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524cf41-76bf-4082-8581-ac7b17eb6722 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:15.919988 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524cf41-76bf-4082-8581-ac7b17eb6722 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-9524cf41-76bf-4082-8581-ac7b17eb6722 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:15.932535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:15.933622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:15.934455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:15.934455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:15.934455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:15.934455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:15.934455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:15.934455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:15.934455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:15.934455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:15.934455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:15.934455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:15.934455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:15.934455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:15.934455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 18:17:15.934455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:15.937323 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 465/898] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:17:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:15.945864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23401c6-3d08-4782-a498-0a13e0b41ed6 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:16.044090 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23401c6-3d08-4782-a498-0a13e0b41ed6 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:16.078462 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7dc4d-c522-4480-a3c8-139012388221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:16.079197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7dc4d-c522-4480-a3c8-139012388221 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:16.079494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7dc4d-c522-4480-a3c8-139012388221 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:16.079774 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7dc4d-c522-4480-a3c8-139012388221 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:16.080494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7dc4d-c522-4480-a3c8-139012388221 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:16.080802 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7dc4d-c522-4480-a3c8-139012388221 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:16.082171 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7dc4d-c522-4480-a3c8-139012388221 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:16.091525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 434/899] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:17:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:16.093057 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524cf41-76bf-4082-8581-ac7b17eb6722 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:16.196407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524cf41-76bf-4082-8581-ac7b17eb6722 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:16.230853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099fbbd-3c88-4f6e-bc55-1d892a3eef85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:16.231540 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099fbbd-3c88-4f6e-bc55-1d892a3eef85 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:16.231893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099fbbd-3c88-4f6e-bc55-1d892a3eef85 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:16.232164 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099fbbd-3c88-4f6e-bc55-1d892a3eef85 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:16.232856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099fbbd-3c88-4f6e-bc55-1d892a3eef85 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:16.233152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099fbbd-3c88-4f6e-bc55-1d892a3eef85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:16.234484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099fbbd-3c88-4f6e-bc55-1d892a3eef85 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:16.240137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 466/900] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:17:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:16.245781 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7dc4d-c522-4480-a3c8-139012388221 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:16.351563 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7dc4d-c522-4480-a3c8-139012388221 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:16.387450 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813094e-1a67-4859-9e87-23e41c32f053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:16.388578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813094e-1a67-4859-9e87-23e41c32f053 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:16.389069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813094e-1a67-4859-9e87-23e41c32f053 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:16.389554 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813094e-1a67-4859-9e87-23e41c32f053 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:16.390613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813094e-1a67-4859-9e87-23e41c32f053 None admin] RBAC: Adding query filter params (interface=public, service_id=a228ca4b392a4af4936138f120ea542a, region_id=RegionOne) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:16.391715 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813094e-1a67-4859-9e87-23e41c32f053 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:16.393346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813094e-1a67-4859-9e87-23e41c32f053 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:16.402534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 435/901] 10.222.0.25 () {56 vars in 1251 bytes} [Mon May 9 18:17:16 2022] GET /identity/v3/endpoints?service_id=a228ca4b392a4af4936138f120ea542a&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:16.409087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099fbbd-3c88-4f6e-bc55-1d892a3eef85 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:16.489229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eabcdf-9460-42b1-bff9-975ae4033dc5 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:16.489561 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eabcdf-9460-42b1-bff9-975ae4033dc5 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:16.489794 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eabcdf-9460-42b1-bff9-975ae4033dc5 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:16.491415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 436/902] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:17:16.496430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78011b-74eb-4bb9-920c-dc4838dc5578 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:16.496666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78011b-74eb-4bb9-920c-dc4838dc5578 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:16.496911 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78011b-74eb-4bb9-920c-dc4838dc5578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:16.513386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-cb78011b-74eb-4bb9-920c-dc4838dc5578 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:16.514746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-cb78011b-74eb-4bb9-920c-dc4838dc5578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:16.519149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099fbbd-3c88-4f6e-bc55-1d892a3eef85 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:16.561300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27de78-9f56-48df-9945-803bd91ad7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:16.562505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27de78-9f56-48df-9945-803bd91ad7f7 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:16.563403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27de78-9f56-48df-9945-803bd91ad7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:16.564034 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27de78-9f56-48df-9945-803bd91ad7f7 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:16.564989 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27de78-9f56-48df-9945-803bd91ad7f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:16.567001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27de78-9f56-48df-9945-803bd91ad7f7 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:16.573683 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 467/903] 10.222.0.25 () {56 vars in 1087 bytes} [Mon May 9 18:17:16 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:17:16.581084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980bfa9-9900-403a-9533-48558f8dd273 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:16.581567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980bfa9-9900-403a-9533-48558f8dd273 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:16.581860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980bfa9-9900-403a-9533-48558f8dd273 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:16.583572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 468/904] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:16.587550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 437/905] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:16.589077 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f13e7-9207-4b95-8b31-df1c20b1e713 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:16.589362 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f13e7-9207-4b95-8b31-df1c20b1e713 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:16.589637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f13e7-9207-4b95-8b31-df1c20b1e713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:16.608033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-567f13e7-9207-4b95-8b31-df1c20b1e713 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:16.609268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-567f13e7-9207-4b95-8b31-df1c20b1e713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:16.611204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78011b-74eb-4bb9-920c-dc4838dc5578 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:16.701211 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 469/906] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:16.726596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f13e7-9207-4b95-8b31-df1c20b1e713 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:16.734432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78011b-74eb-4bb9-920c-dc4838dc5578 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:16.778850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ec196-eb22-4db5-8193-e31edaaa4253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:16.779506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ec196-eb22-4db5-8193-e31edaaa4253 None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:16.779762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ec196-eb22-4db5-8193-e31edaaa4253 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:16.780673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ec196-eb22-4db5-8193-e31edaaa4253 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:16.780801 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ec196-eb22-4db5-8193-e31edaaa4253 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:16.782144 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ec196-eb22-4db5-8193-e31edaaa4253 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:16.798202 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 438/907] 10.222.0.25 () {60 vars in 1141 bytes} [Mon May 9 18:17:16 2022] POST /identity/v3/services => generated 241 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:17:16.849255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f13e7-9207-4b95-8b31-df1c20b1e713 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:16.893865 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54ead6-bca8-494c-ab04-af7075ff7db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:16.895097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54ead6-bca8-494c-ab04-af7075ff7db6 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:16.895614 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54ead6-bca8-494c-ab04-af7075ff7db6 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:16.896090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54ead6-bca8-494c-ab04-af7075ff7db6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:16.896951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54ead6-bca8-494c-ab04-af7075ff7db6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:16.898532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54ead6-bca8-494c-ab04-af7075ff7db6 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-9e54ead6-bca8-494c-ab04-af7075ff7db6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:16.911752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:16.912787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 18:17:16.913636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:16.916720 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 470/908] 10.222.0.25 () {56 vars in 1132 bytes} [Mon May 9 18:17:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:16.924360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ec196-eb22-4db5-8193-e31edaaa4253 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:17.033212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ec196-eb22-4db5-8193-e31edaaa4253 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:17.067398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412511c-8477-42dd-b408-778d38d3b728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:17.068073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412511c-8477-42dd-b408-778d38d3b728 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:17.068355 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412511c-8477-42dd-b408-778d38d3b728 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:17.068631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412511c-8477-42dd-b408-778d38d3b728 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:17.069391 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412511c-8477-42dd-b408-778d38d3b728 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:17.069709 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412511c-8477-42dd-b408-778d38d3b728 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:17.071073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412511c-8477-42dd-b408-778d38d3b728 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:17.076183 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 439/909] 10.222.0.25 () {56 vars in 1126 bytes} [Mon May 9 18:17:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:17.081963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54ead6-bca8-494c-ab04-af7075ff7db6 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:17.180831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54ead6-bca8-494c-ab04-af7075ff7db6 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:17.212855 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223560c9-1db6-4fe0-8d2f-a9513b6b0cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:17.213545 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223560c9-1db6-4fe0-8d2f-a9513b6b0cd4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:17.213791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223560c9-1db6-4fe0-8d2f-a9513b6b0cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:17.214065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223560c9-1db6-4fe0-8d2f-a9513b6b0cd4 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:17.214754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223560c9-1db6-4fe0-8d2f-a9513b6b0cd4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:17.215054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223560c9-1db6-4fe0-8d2f-a9513b6b0cd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:17.216439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223560c9-1db6-4fe0-8d2f-a9513b6b0cd4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:17.221530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 471/910] 10.222.0.25 () {56 vars in 1126 bytes} [Mon May 9 18:17:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:17.227346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412511c-8477-42dd-b408-778d38d3b728 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:17.327976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412511c-8477-42dd-b408-778d38d3b728 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:17.363538 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd455f-6ace-4214-a3fd-f3ed6c8a0d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:17.364715 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd455f-6ace-4214-a3fd-f3ed6c8a0d9c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:17.365590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd455f-6ace-4214-a3fd-f3ed6c8a0d9c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:17.366455 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd455f-6ace-4214-a3fd-f3ed6c8a0d9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:17.368244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd455f-6ace-4214-a3fd-f3ed6c8a0d9c None admin] RBAC: Adding query filter params (interface=public, service_id=151dbe05b4774f1d918187967d49465c, region_id=RegionOne) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:17.368915 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd455f-6ace-4214-a3fd-f3ed6c8a0d9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:17.370489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd455f-6ace-4214-a3fd-f3ed6c8a0d9c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:17.377539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 440/911] 10.222.0.25 () {56 vars in 1251 bytes} [Mon May 9 18:17:17 2022] GET /identity/v3/endpoints?service_id=151dbe05b4774f1d918187967d49465c&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:17.384510 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223560c9-1db6-4fe0-8d2f-a9513b6b0cd4 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:17.484104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223560c9-1db6-4fe0-8d2f-a9513b6b0cd4 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:17.518045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73fa52-5c98-47ef-ac94-1df52a9d867a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:17.518708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73fa52-5c98-47ef-ac94-1df52a9d867a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:17.518986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73fa52-5c98-47ef-ac94-1df52a9d867a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:17.519285 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73fa52-5c98-47ef-ac94-1df52a9d867a None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:17.519960 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73fa52-5c98-47ef-ac94-1df52a9d867a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:17.521316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73fa52-5c98-47ef-ac94-1df52a9d867a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:17.527514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 472/912] 10.222.0.25 () {56 vars in 1087 bytes} [Mon May 9 18:17:17 2022] GET /identity/v3/services => generated 1497 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:17:17.951402 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15bc72-8f44-441b-bb79-d3cac28ac2f4 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:17.951786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15bc72-8f44-441b-bb79-d3cac28ac2f4 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:17.951965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15bc72-8f44-441b-bb79-d3cac28ac2f4 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:17.953613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 441/913] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:17.959617 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b415a-8484-41c5-87f8-54713be5aac8 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:17.961591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b415a-8484-41c5-87f8-54713be5aac8 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:17.962095 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b415a-8484-41c5-87f8-54713be5aac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:17.986996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4a9b415a-8484-41c5-87f8-54713be5aac8 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:17.988201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4a9b415a-8484-41c5-87f8-54713be5aac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:18.056551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 473/914] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:18.078946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15bc72-8f44-441b-bb79-d3cac28ac2f4 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:18.179484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15bc72-8f44-441b-bb79-d3cac28ac2f4 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:18.213428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323b410-b452-4f5b-bb96-d16bee287197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:18.214582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323b410-b452-4f5b-bb96-d16bee287197 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:18.215069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323b410-b452-4f5b-bb96-d16bee287197 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:18.215576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323b410-b452-4f5b-bb96-d16bee287197 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:18.227065 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323b410-b452-4f5b-bb96-d16bee287197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-6323b410-b452-4f5b-bb96-d16bee287197 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:18.236092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:18.237076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:18.237955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:18.237955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:18.237955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:18.237955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:18.237955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:18.237955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:18.237955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:18.237955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:18.237955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:18.237955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:18.237955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 18:17:18.237955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:18.245144 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 442/915] 10.222.0.25 () {56 vars in 1120 bytes} [Mon May 9 18:17:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:18.248267 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b415a-8484-41c5-87f8-54713be5aac8 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:18.343924 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b415a-8484-41c5-87f8-54713be5aac8 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:18.384155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e578e7e-52cb-4620-a2a4-9775a3ddf58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:18.384821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e578e7e-52cb-4620-a2a4-9775a3ddf58d None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:18.385099 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e578e7e-52cb-4620-a2a4-9775a3ddf58d None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:18.385366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e578e7e-52cb-4620-a2a4-9775a3ddf58d None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:18.386097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e578e7e-52cb-4620-a2a4-9775a3ddf58d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:18.386397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e578e7e-52cb-4620-a2a4-9775a3ddf58d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:18.387788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e578e7e-52cb-4620-a2a4-9775a3ddf58d None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:18.396368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 474/916] 10.222.0.25 () {56 vars in 1115 bytes} [Mon May 9 18:17:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:18.404500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323b410-b452-4f5b-bb96-d16bee287197 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:18.504071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323b410-b452-4f5b-bb96-d16bee287197 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:18.537830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69214f33-ae34-42d2-9081-6fc76c117e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:18.538970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69214f33-ae34-42d2-9081-6fc76c117e7f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:18.539496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69214f33-ae34-42d2-9081-6fc76c117e7f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:18.539976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69214f33-ae34-42d2-9081-6fc76c117e7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:18.550853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69214f33-ae34-42d2-9081-6fc76c117e7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:18.552584 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69214f33-ae34-42d2-9081-6fc76c117e7f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-69214f33-ae34-42d2-9081-6fc76c117e7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:18.570035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:18.570974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:18.571887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:18.571887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:18.571887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:18.571887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:18.578410 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 443/917] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:18.584260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e578e7e-52cb-4620-a2a4-9775a3ddf58d None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:18.684356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e578e7e-52cb-4620-a2a4-9775a3ddf58d None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:18.720345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e2156-6731-4052-97a2-4282dbcf91a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:18.721021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e2156-6731-4052-97a2-4282dbcf91a0 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:18.721299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e2156-6731-4052-97a2-4282dbcf91a0 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:18.721569 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e2156-6731-4052-97a2-4282dbcf91a0 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:18.722289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e2156-6731-4052-97a2-4282dbcf91a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:18.722590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e2156-6731-4052-97a2-4282dbcf91a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:18.723974 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e2156-6731-4052-97a2-4282dbcf91a0 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:18.739484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 475/918] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:18.745517 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69214f33-ae34-42d2-9081-6fc76c117e7f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:18.850402 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69214f33-ae34-42d2-9081-6fc76c117e7f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:18.883826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec123a36-56ea-4358-8459-2dcc5552cda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:18.885068 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec123a36-56ea-4358-8459-2dcc5552cda6 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:18.885556 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec123a36-56ea-4358-8459-2dcc5552cda6 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:18.886035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec123a36-56ea-4358-8459-2dcc5552cda6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:18.894359 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec123a36-56ea-4358-8459-2dcc5552cda6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:18.896079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec123a36-56ea-4358-8459-2dcc5552cda6 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-ec123a36-56ea-4358-8459-2dcc5552cda6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:17:18.911485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:18.912478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:18.913428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:18.913428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:18.913428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:17:18.913428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:17:18.913428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:17:18.913428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:17:18.913428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:17:18.913428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:17:18.913428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 18:17:18.913428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:18.920364 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 444/919] 10.222.0.25 () {56 vars in 1117 bytes} [Mon May 9 18:17:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:18.921907 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e2156-6731-4052-97a2-4282dbcf91a0 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:18.954766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7a56f-9dc1-448b-965f-71b826cabf83 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:18.955437 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7a56f-9dc1-448b-965f-71b826cabf83 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:18.955918 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7a56f-9dc1-448b-965f-71b826cabf83 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:18.958145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 445/920] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:18.964544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f31a2b-c7b3-41aa-8995-bd0578c09cb5 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:18.965049 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f31a2b-c7b3-41aa-8995-bd0578c09cb5 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:18.965533 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f31a2b-c7b3-41aa-8995-bd0578c09cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:18.983525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-e8f31a2b-c7b3-41aa-8995-bd0578c09cb5 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:18.985046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e8f31a2b-c7b3-41aa-8995-bd0578c09cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:19.040416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e2156-6731-4052-97a2-4282dbcf91a0 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:19.072719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 446/921] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:19.079909 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312ac13-1987-42f9-b3b7-13f8f7a04d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:19.080756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312ac13-1987-42f9-b3b7-13f8f7a04d59 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:19.081170 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312ac13-1987-42f9-b3b7-13f8f7a04d59 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:19.081574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312ac13-1987-42f9-b3b7-13f8f7a04d59 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:19.082642 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312ac13-1987-42f9-b3b7-13f8f7a04d59 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:19.083138 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312ac13-1987-42f9-b3b7-13f8f7a04d59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:19.084750 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312ac13-1987-42f9-b3b7-13f8f7a04d59 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:19.096720 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f31a2b-c7b3-41aa-8995-bd0578c09cb5 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:19.106832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 476/922] 10.222.0.25 () {56 vars in 1149 bytes} [Mon May 9 18:17:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:19.108437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312ac13-1987-42f9-b3b7-13f8f7a04d59 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:19.202800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f31a2b-c7b3-41aa-8995-bd0578c09cb5 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:19.227387 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312ac13-1987-42f9-b3b7-13f8f7a04d59 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:19.236734 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25b764-d138-408c-8b1e-119917ebd7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:19.237540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25b764-d138-408c-8b1e-119917ebd7ff None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:19.237827 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25b764-d138-408c-8b1e-119917ebd7ff None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:19.238105 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25b764-d138-408c-8b1e-119917ebd7ff None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:19.238750 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25b764-d138-408c-8b1e-119917ebd7ff None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:19.240132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25b764-d138-408c-8b1e-119917ebd7ff None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-7c25b764-d138-408c-8b1e-119917ebd7ff None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:19.252485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:19.253541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:19.259863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 447/923] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:17:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:19.260666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4776af2-50e1-4af7-8811-493ad8c4d8c9 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:19.260666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4776af2-50e1-4af7-8811-493ad8c4d8c9 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:19.260666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4776af2-50e1-4af7-8811-493ad8c4d8c9 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:19.261267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 448/924] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:19.262016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4776af2-50e1-4af7-8811-493ad8c4d8c9 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:19.262587 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689530c-b50d-4c12-ade0-6256deec9ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:19.263417 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689530c-b50d-4c12-ade0-6256deec9ec6 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:19.263697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689530c-b50d-4c12-ade0-6256deec9ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:19.263966 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689530c-b50d-4c12-ade0-6256deec9ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:19.274591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689530c-b50d-4c12-ade0-6256deec9ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:19.276059 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689530c-b50d-4c12-ade0-6256deec9ec6 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-f689530c-b50d-4c12-ade0-6256deec9ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:19.293043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:19.293978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:19.294902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:19.294902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:19.294902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:19.294902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:19.294902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 477/925] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:19.295682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84d42a-63ce-4b9e-8462-77a26615a4e3 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:19.295682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84d42a-63ce-4b9e-8462-77a26615a4e3 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:19.295946 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84d42a-63ce-4b9e-8462-77a26615a4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:19.311430 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-8c84d42a-63ce-4b9e-8462-77a26615a4e3 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:19.312589 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-8c84d42a-63ce-4b9e-8462-77a26615a4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:19.368946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4776af2-50e1-4af7-8811-493ad8c4d8c9 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:19.386210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 478/926] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:19.387060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84d42a-63ce-4b9e-8462-77a26615a4e3 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:19.411400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ff86b-d529-4b9d-ac45-8a9072e1fbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:19.413204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ff86b-d529-4b9d-ac45-8a9072e1fbc5 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:19.413661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ff86b-d529-4b9d-ac45-8a9072e1fbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:19.414263 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ff86b-d529-4b9d-ac45-8a9072e1fbc5 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:19.415459 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ff86b-d529-4b9d-ac45-8a9072e1fbc5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:19.416159 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ff86b-d529-4b9d-ac45-8a9072e1fbc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:19.418136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ff86b-d529-4b9d-ac45-8a9072e1fbc5 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:19.429228 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 449/927] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:17:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:19.429228 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ff86b-d529-4b9d-ac45-8a9072e1fbc5 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:19.496460 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84d42a-63ce-4b9e-8462-77a26615a4e3 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:19.525298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ff86b-d529-4b9d-ac45-8a9072e1fbc5 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:19.530606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eece677-582e-491f-a2fd-df7a2be0dc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:19.531298 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eece677-582e-491f-a2fd-df7a2be0dc61 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:19.531723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eece677-582e-491f-a2fd-df7a2be0dc61 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:19.532003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eece677-582e-491f-a2fd-df7a2be0dc61 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:19.532717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eece677-582e-491f-a2fd-df7a2be0dc61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:19.533022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eece677-582e-491f-a2fd-df7a2be0dc61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:19.534378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eece677-582e-491f-a2fd-df7a2be0dc61 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:19.550863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 479/928] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:19.551796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eece677-582e-491f-a2fd-df7a2be0dc61 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:19.559301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1a970-9bab-44e2-b232-d575f239a055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:19.560121 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1a970-9bab-44e2-b232-d575f239a055 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:19.560406 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1a970-9bab-44e2-b232-d575f239a055 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:19.560683 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1a970-9bab-44e2-b232-d575f239a055 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:19.561328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1a970-9bab-44e2-b232-d575f239a055 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:19.562682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1a970-9bab-44e2-b232-d575f239a055 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-b3e1a970-9bab-44e2-b232-d575f239a055 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:19.575285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:19.576308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:19.577146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 450/929] 10.222.0.25 () {56 vars in 1126 bytes} [Mon May 9 18:17:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:19.577960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1a970-9bab-44e2-b232-d575f239a055 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:19.651313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eece677-582e-491f-a2fd-df7a2be0dc61 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:19.674371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1a970-9bab-44e2-b232-d575f239a055 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:19.686191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a3510-355c-486b-95d8-c8978a3809ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:19.686884 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a3510-355c-486b-95d8-c8978a3809ed None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:19.687166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a3510-355c-486b-95d8-c8978a3809ed None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:19.687471 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a3510-355c-486b-95d8-c8978a3809ed None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:19.688763 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a3510-355c-486b-95d8-c8978a3809ed None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:19.689070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a3510-355c-486b-95d8-c8978a3809ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:19.690478 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a3510-355c-486b-95d8-c8978a3809ed None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:19.700492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 480/930] 10.222.0.25 () {56 vars in 1114 bytes} [Mon May 9 18:17:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:19.700492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a3510-355c-486b-95d8-c8978a3809ed None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:19.717345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49ce46-5bf7-4044-a275-3fcdace73de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:19.718151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49ce46-5bf7-4044-a275-3fcdace73de3 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:19.718433 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49ce46-5bf7-4044-a275-3fcdace73de3 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:19.718705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49ce46-5bf7-4044-a275-3fcdace73de3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:19.726105 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49ce46-5bf7-4044-a275-3fcdace73de3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:19.727573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49ce46-5bf7-4044-a275-3fcdace73de3 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-1f49ce46-5bf7-4044-a275-3fcdace73de3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:17:19.742178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:19.743294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:19.744204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:17:19.744204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:17:19.744204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:17:19.744204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:17:19.744204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:17:19.744204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:17:19.744204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:19.744204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:19.744204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 451/931] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:19.744204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49ce46-5bf7-4044-a275-3fcdace73de3 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:19.800090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a3510-355c-486b-95d8-c8978a3809ed None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:19.832253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ab969-9228-4fea-a631-7d49ca023f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:19.832912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ab969-9228-4fea-a631-7d49ca023f71 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:19.833195 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ab969-9228-4fea-a631-7d49ca023f71 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:19.833469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ab969-9228-4fea-a631-7d49ca023f71 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:19.834170 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ab969-9228-4fea-a631-7d49ca023f71 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:19.834473 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ab969-9228-4fea-a631-7d49ca023f71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:19.835856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ab969-9228-4fea-a631-7d49ca023f71 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:19.840857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 481/932] 10.222.0.25 () {56 vars in 1122 bytes} [Mon May 9 18:17:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:19.841520 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ab969-9228-4fea-a631-7d49ca023f71 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:19.849070 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49ce46-5bf7-4044-a275-3fcdace73de3 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:19.883097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71217e9-b42b-4fa6-bf07-87bae4313ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:19.883786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71217e9-b42b-4fa6-bf07-87bae4313ed3 None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:19.884064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71217e9-b42b-4fa6-bf07-87bae4313ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:19.884331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71217e9-b42b-4fa6-bf07-87bae4313ed3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:19.885048 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71217e9-b42b-4fa6-bf07-87bae4313ed3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:19.886383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71217e9-b42b-4fa6-bf07-87bae4313ed3 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:19.905313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 452/933] 10.222.0.25 () {60 vars in 1144 bytes} [Mon May 9 18:17:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:17:19.906034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71217e9-b42b-4fa6-bf07-87bae4313ed3 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:19.944103 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ab969-9228-4fea-a631-7d49ca023f71 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:19.988823 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0377d-461c-4946-9072-e39ddfd195ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:19.989581 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0377d-461c-4946-9072-e39ddfd195ae None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:19.989932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0377d-461c-4946-9072-e39ddfd195ae None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:19.990295 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0377d-461c-4946-9072-e39ddfd195ae None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:19.991082 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0377d-461c-4946-9072-e39ddfd195ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:19.991466 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0377d-461c-4946-9072-e39ddfd195ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:19.992845 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0377d-461c-4946-9072-e39ddfd195ae None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:20.007130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 482/934] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:17:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:20.008648 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6b33f-5bbc-4389-b547-2bd1cb7c3835 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:20.008936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6b33f-5bbc-4389-b547-2bd1cb7c3835 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:20.009266 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6b33f-5bbc-4389-b547-2bd1cb7c3835 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:20.015014 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 483/935] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:20.017418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dae87-3240-436b-bb8c-a15ee91964d3 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:20.017835 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dae87-3240-436b-bb8c-a15ee91964d3 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:20.018210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dae87-3240-436b-bb8c-a15ee91964d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:20.018980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71217e9-b42b-4fa6-bf07-87bae4313ed3 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:20.034525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-715dae87-3240-436b-bb8c-a15ee91964d3 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:20.036000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-715dae87-3240-436b-bb8c-a15ee91964d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:20.057728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa72414-f3c7-4df1-960b-f9e1c84961e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:20.058436 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa72414-f3c7-4df1-960b-f9e1c84961e2 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:20.058714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa72414-f3c7-4df1-960b-f9e1c84961e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:20.058981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa72414-f3c7-4df1-960b-f9e1c84961e2 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:20.059724 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa72414-f3c7-4df1-960b-f9e1c84961e2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:20.060026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa72414-f3c7-4df1-960b-f9e1c84961e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:20.061398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa72414-f3c7-4df1-960b-f9e1c84961e2 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:20.067608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 453/936] 10.222.0.25 () {56 vars in 1122 bytes} [Mon May 9 18:17:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:20.068503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa72414-f3c7-4df1-960b-f9e1c84961e2 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:20.114215 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 484/937] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:20.115423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dae87-3240-436b-bb8c-a15ee91964d3 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:20.171015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa72414-f3c7-4df1-960b-f9e1c84961e2 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:20.204350 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f172d65-f23d-45be-bdba-89fb2beb3482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:20.205245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f172d65-f23d-45be-bdba-89fb2beb3482 None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:20.205524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f172d65-f23d-45be-bdba-89fb2beb3482 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:20.205809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f172d65-f23d-45be-bdba-89fb2beb3482 None admin] PATH_INFO: `/v3/projects/4849b58351c74caf92a584686c74e397/users/154f86b7575e47efb2a5e1c35943e037/roles/104650f78a264c78a8cf5c6c4b389575` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:20.208112 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f172d65-f23d-45be-bdba-89fb2beb3482 None admin] RBAC: Authorizing `identity:create_grant(project_id=4849b58351c74caf92a584686c74e397, user_id=154f86b7575e47efb2a5e1c35943e037, role_id=104650f78a264c78a8cf5c6c4b389575)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:20.209530 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f172d65-f23d-45be-bdba-89fb2beb3482 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:20.231205 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 454/938] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:17:20 2022] PUT /identity/v3/projects/4849b58351c74caf92a584686c74e397/users/154f86b7575e47efb2a5e1c35943e037/roles/104650f78a264c78a8cf5c6c4b389575 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:17:20.231447 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dae87-3240-436b-bb8c-a15ee91964d3 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:20.231545 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f172d65-f23d-45be-bdba-89fb2beb3482 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:20.261257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6a9fe-df28-4d71-af5a-3416883feba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:20.261939 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6a9fe-df28-4d71-af5a-3416883feba1 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:20.262210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6a9fe-df28-4d71-af5a-3416883feba1 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:20.262482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6a9fe-df28-4d71-af5a-3416883feba1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:20.263311 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6a9fe-df28-4d71-af5a-3416883feba1 None admin] RBAC: Adding query filter params (interface=public, service_id=f3678b4b0d3841b39a5e3e7c59224083, region_id=RegionOne) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:20.263852 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6a9fe-df28-4d71-af5a-3416883feba1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:20.264995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6a9fe-df28-4d71-af5a-3416883feba1 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:20.271028 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 485/939] 10.222.0.25 () {56 vars in 1251 bytes} [Mon May 9 18:17:20 2022] GET /identity/v3/endpoints?service_id=f3678b4b0d3841b39a5e3e7c59224083&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:20.276467 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6a9fe-df28-4d71-af5a-3416883feba1 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:20.345256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f172d65-f23d-45be-bdba-89fb2beb3482 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:20.381957 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff391e-cd02-4bcc-9f1b-33eb10c6fbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:20.382837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff391e-cd02-4bcc-9f1b-33eb10c6fbe6 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:20.383121 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff391e-cd02-4bcc-9f1b-33eb10c6fbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:20.383252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6a9fe-df28-4d71-af5a-3416883feba1 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:20.383439 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff391e-cd02-4bcc-9f1b-33eb10c6fbe6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:20.384093 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff391e-cd02-4bcc-9f1b-33eb10c6fbe6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:20.385457 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff391e-cd02-4bcc-9f1b-33eb10c6fbe6 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-dcff391e-cd02-4bcc-9f1b-33eb10c6fbe6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:20.399017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:20.400452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:20.401777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 455/940] 10.222.0.25 () {56 vars in 1132 bytes} [Mon May 9 18:17:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:20.407371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff391e-cd02-4bcc-9f1b-33eb10c6fbe6 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:20.428108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7929e-b8b2-4439-8d94-5991defb55d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:20.428698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7929e-b8b2-4439-8d94-5991defb55d1 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:20.428976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7929e-b8b2-4439-8d94-5991defb55d1 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:20.429245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7929e-b8b2-4439-8d94-5991defb55d1 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:20.429918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7929e-b8b2-4439-8d94-5991defb55d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:20.431306 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7929e-b8b2-4439-8d94-5991defb55d1 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:20.437855 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 486/941] 10.222.0.25 () {56 vars in 1087 bytes} [Mon May 9 18:17:20 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:17:20.519234 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff391e-cd02-4bcc-9f1b-33eb10c6fbe6 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:20.556102 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2dc94-d9cc-4460-8eac-6a5783e48ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:20.556790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2dc94-d9cc-4460-8eac-6a5783e48ae5 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:20.557086 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2dc94-d9cc-4460-8eac-6a5783e48ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:20.557346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2dc94-d9cc-4460-8eac-6a5783e48ae5 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:20.558063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2dc94-d9cc-4460-8eac-6a5783e48ae5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:20.558370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2dc94-d9cc-4460-8eac-6a5783e48ae5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:20.559788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2dc94-d9cc-4460-8eac-6a5783e48ae5 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:20.566027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 456/942] 10.222.0.25 () {56 vars in 1126 bytes} [Mon May 9 18:17:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:20.571846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7929e-b8b2-4439-8d94-5991defb55d1 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:20.678815 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7929e-b8b2-4439-8d94-5991defb55d1 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:20.713947 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a59e3-12ea-4335-ab09-f4fa851104e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:20.714995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a59e3-12ea-4335-ab09-f4fa851104e9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:20.715514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a59e3-12ea-4335-ab09-f4fa851104e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:20.716031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a59e3-12ea-4335-ab09-f4fa851104e9 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:20.716940 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a59e3-12ea-4335-ab09-f4fa851104e9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:20.717459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a59e3-12ea-4335-ab09-f4fa851104e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:20.719020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a59e3-12ea-4335-ab09-f4fa851104e9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:20.729521 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 487/943] 10.222.0.25 () {56 vars in 1126 bytes} [Mon May 9 18:17:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:20.735050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2dc94-d9cc-4460-8eac-6a5783e48ae5 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:20.840944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2dc94-d9cc-4460-8eac-6a5783e48ae5 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:20.874766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19057c4-eb77-4afc-a70c-acf6f41ed22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:20.876022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19057c4-eb77-4afc-a70c-acf6f41ed22f None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:20.876312 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19057c4-eb77-4afc-a70c-acf6f41ed22f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:20.876589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19057c4-eb77-4afc-a70c-acf6f41ed22f None admin] PATH_INFO: `/v3/endpoints` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:20.877589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19057c4-eb77-4afc-a70c-acf6f41ed22f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:20.878938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19057c4-eb77-4afc-a70c-acf6f41ed22f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:20.899954 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 457/944] 10.222.0.25 () {60 vars in 1144 bytes} [Mon May 9 18:17:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:17:22.221063 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3220f-6ba9-4633-912d-3cf9e4f85e34 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:22.221560 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3220f-6ba9-4633-912d-3cf9e4f85e34 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:22.221560 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3220f-6ba9-4633-912d-3cf9e4f85e34 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:22.223158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 488/945] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:22.228685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de720098-d167-4222-829c-f7713f8a03d4 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:22.229182 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de720098-d167-4222-829c-f7713f8a03d4 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:22.229362 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de720098-d167-4222-829c-f7713f8a03d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:22.246133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-de720098-d167-4222-829c-f7713f8a03d4 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:22.247334 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-de720098-d167-4222-829c-f7713f8a03d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:22.327294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 458/946] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:22.347857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3220f-6ba9-4633-912d-3cf9e4f85e34 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:22.455032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3220f-6ba9-4633-912d-3cf9e4f85e34 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:22.491555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53bc23-1266-4025-b1b2-e548c1cb620e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:22.492559 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53bc23-1266-4025-b1b2-e548c1cb620e None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:22.493041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53bc23-1266-4025-b1b2-e548c1cb620e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:22.493549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53bc23-1266-4025-b1b2-e548c1cb620e None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:22.494466 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c53bc23-1266-4025-b1b2-e548c1cb620e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:22.496091 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c53bc23-1266-4025-b1b2-e548c1cb620e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:22.497661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: WARNING keystone.server.flask.common [None req-7c53bc23-1266-4025-b1b2-e548c1cb620e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 09 18:17:22.515908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.sql.core [None req-7c53bc23-1266-4025-b1b2-e548c1cb620e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 18:17:22.515908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 18:17:22.515908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [parameters: {'id': 'e7661b5c0e9f4da4a0c7e479944bc628', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 18:17:22.515908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79826) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 18:17:22.533263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-7c53bc23-1266-4025-b1b2-e548c1cb620e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e7661b5c0e9f4da4a0c7e479944bc628.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e7661b5c0e9f4da4a0c7e479944bc628. May 09 18:17:22.533263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:22.533263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 18:17:22.533263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 18:17:22.533263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 18:17:22.533263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 18:17:22.533263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 18:17:22.533263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application result = self._query(query) May 09 18:17:22.533263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 18:17:22.533263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application conn.query(q) May 09 18:17:22.533263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 18:17:22.533263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 18:17:22.533263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application result.read() May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 09 18:17:22.534113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return project_ref.to_dict() May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application next(self.gen) May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application yield resource May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application next(self.gen) May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self.session.rollback() May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self.force_reraise() May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise self.value May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application session.commit() May 09 18:17:22.535165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._prepare_impl() May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self.session.flush() May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._flush(objects) May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application compat.raise_( May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application flush_context.execute() May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 18:17:22.536107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rec.execute(self) May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 18:17:22.537890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application result = self._query(query) May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application conn.query(q) May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application result.read() May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 18:17:22.539877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [parameters: {'id': 'e7661b5c0e9f4da4a0c7e479944bc628', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:22.541982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:22.543870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:22.546201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:22.546201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 09 18:17:22.546201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 09 18:17:22.546201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:22.546201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:22.546201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 09 18:17:22.546201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.Conflict( May 09 18:17:22.546201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e7661b5c0e9f4da4a0c7e479944bc628. May 09 18:17:22.546201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:22.550375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 489/947] 10.222.0.25 () {60 vars in 1140 bytes} [Mon May 9 18:17:22 2022] POST /identity/v3/projects => generated 262 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:22.556540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de720098-d167-4222-829c-f7713f8a03d4 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:22.629242 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b8b75-b795-4a94-9798-f6c6c50bc9e0 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:22.629522 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b8b75-b795-4a94-9798-f6c6c50bc9e0 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:22.629797 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b8b75-b795-4a94-9798-f6c6c50bc9e0 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:22.634136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 490/948] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:22.640683 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62a911-ce8c-4d2d-bc3a-b5378287370d None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:22.641229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62a911-ce8c-4d2d-bc3a-b5378287370d None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:22.642086 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62a911-ce8c-4d2d-bc3a-b5378287370d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:22.665935 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-6a62a911-ce8c-4d2d-bc3a-b5378287370d None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:22.666451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de720098-d167-4222-829c-f7713f8a03d4 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:22.668119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-6a62a911-ce8c-4d2d-bc3a-b5378287370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:22.712660 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a35b4e-52f7-4d60-8c63-75364d56d5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:22.713559 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a35b4e-52f7-4d60-8c63-75364d56d5d2 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:22.713907 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a35b4e-52f7-4d60-8c63-75364d56d5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:22.714962 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a35b4e-52f7-4d60-8c63-75364d56d5d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:22.721940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a35b4e-52f7-4d60-8c63-75364d56d5d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:22.723497 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a35b4e-52f7-4d60-8c63-75364d56d5d2 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-a4a35b4e-52f7-4d60-8c63-75364d56d5d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:17:22.737739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:22.738864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:22.740034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:17:22.740034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:17:22.740034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:17:22.740034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:17:22.740034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:17:22.740034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:17:22.740034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:22.740034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:22.745557 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 459/949] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:22.746807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a35b4e-52f7-4d60-8c63-75364d56d5d2 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:22.753838 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 491/950] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:22.755453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f352a-fcf5-47f1-98a7-5e9df889d2d0 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:22.755916 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f352a-fcf5-47f1-98a7-5e9df889d2d0 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:22.756342 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f352a-fcf5-47f1-98a7-5e9df889d2d0 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:22.758001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 492/951] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:22.764619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7f2e2-047a-431b-8da8-bf2625e6daf7 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:22.765283 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7f2e2-047a-431b-8da8-bf2625e6daf7 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:22.765576 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7f2e2-047a-431b-8da8-bf2625e6daf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:22.785903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c8c7f2e2-047a-431b-8da8-bf2625e6daf7 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:22.786835 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c8c7f2e2-047a-431b-8da8-bf2625e6daf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:22.852013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a35b4e-52f7-4d60-8c63-75364d56d5d2 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:22.854159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 493/952] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:22.854992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7f2e2-047a-431b-8da8-bf2625e6daf7 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:22.889689 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d1d81-7a4a-48d8-b506-89e2070a9f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:22.890346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d1d81-7a4a-48d8-b506-89e2070a9f97 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:22.890629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d1d81-7a4a-48d8-b506-89e2070a9f97 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:22.890937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d1d81-7a4a-48d8-b506-89e2070a9f97 None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:22.891709 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d1d81-7a4a-48d8-b506-89e2070a9f97 None admin] RBAC: Adding query filter params (name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:22.892007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d1d81-7a4a-48d8-b506-89e2070a9f97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:22.893414 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d1d81-7a4a-48d8-b506-89e2070a9f97 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:22.910646 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 460/953] 10.222.0.25 () {56 vars in 1112 bytes} [Mon May 9 18:17:22 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:22.911726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d1d81-7a4a-48d8-b506-89e2070a9f97 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:22.973613 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7f2e2-047a-431b-8da8-bf2625e6daf7 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:23.021223 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede1e09-6d0b-4893-9128-707a3a635ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:23.022086 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede1e09-6d0b-4893-9128-707a3a635ed5 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:23.022959 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede1e09-6d0b-4893-9128-707a3a635ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:23.023368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede1e09-6d0b-4893-9128-707a3a635ed5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:23.039286 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede1e09-6d0b-4893-9128-707a3a635ed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-8ede1e09-6d0b-4893-9128-707a3a635ed5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:23.051031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:23.052289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:23.053351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:23.053351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:23.053351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:23.053351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:23.053351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:23.053351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:23.053351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:23.053351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:23.053351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:23.053351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:23.053351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 18:17:23.053351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:23.054569 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d1d81-7a4a-48d8-b506-89e2070a9f97 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:23.054949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 494/954] 10.222.0.25 () {56 vars in 1120 bytes} [Mon May 9 18:17:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:23.060420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede1e09-6d0b-4893-9128-707a3a635ed5 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:23.099677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9c423-1b84-4193-8502-0671d306a82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:23.100897 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9c423-1b84-4193-8502-0671d306a82d None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:23.101408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9c423-1b84-4193-8502-0671d306a82d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:23.101980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9c423-1b84-4193-8502-0671d306a82d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:23.102864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9c423-1b84-4193-8502-0671d306a82d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:23.104530 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9c423-1b84-4193-8502-0671d306a82d None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-16d9c423-1b84-4193-8502-0671d306a82d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:23.117574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:23.118661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:23.119540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:23.119540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:23.119540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:23.119540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:23.119540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:23.119540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:23.119540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:23.119540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:23.119540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:23.119540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:23.119540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:23.119540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:23.119540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:23.119540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:23.121365 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 461/955] 10.222.0.25 () {56 vars in 1126 bytes} [Mon May 9 18:17:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:23.127741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9c423-1b84-4193-8502-0671d306a82d None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:23.182091 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede1e09-6d0b-4893-9128-707a3a635ed5 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:23.221802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20116b-5c2a-4a60-99eb-09dc1a51d0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:23.222556 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20116b-5c2a-4a60-99eb-09dc1a51d0b9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:23.222925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20116b-5c2a-4a60-99eb-09dc1a51d0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:23.223302 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20116b-5c2a-4a60-99eb-09dc1a51d0b9 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:23.224060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20116b-5c2a-4a60-99eb-09dc1a51d0b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:23.224364 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20116b-5c2a-4a60-99eb-09dc1a51d0b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:23.225861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20116b-5c2a-4a60-99eb-09dc1a51d0b9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:23.234113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 495/956] 10.222.0.25 () {56 vars in 1115 bytes} [Mon May 9 18:17:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:23.238753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508f7dd-864a-455e-a4ef-2c597e9cf557 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:23.238753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508f7dd-864a-455e-a4ef-2c597e9cf557 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:23.238753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508f7dd-864a-455e-a4ef-2c597e9cf557 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:23.238753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 496/957] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:23.239716 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508f7dd-864a-455e-a4ef-2c597e9cf557 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:23.243999 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9c423-1b84-4193-8502-0671d306a82d None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:23.277416 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e69cf-00d0-4dfb-9478-4b07ec590d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:23.278013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e69cf-00d0-4dfb-9478-4b07ec590d80 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:23.278299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e69cf-00d0-4dfb-9478-4b07ec590d80 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:23.278579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e69cf-00d0-4dfb-9478-4b07ec590d80 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:23.279308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e69cf-00d0-4dfb-9478-4b07ec590d80 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:23.279618 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e69cf-00d0-4dfb-9478-4b07ec590d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:23.281080 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e69cf-00d0-4dfb-9478-4b07ec590d80 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:23.286252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 462/958] 10.222.0.25 () {56 vars in 1122 bytes} [Mon May 9 18:17:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:23.287807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deea611-7006-4cb7-831a-822b209df03d None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:23.288307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deea611-7006-4cb7-831a-822b209df03d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:23.288791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deea611-7006-4cb7-831a-822b209df03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:23.305182 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-6deea611-7006-4cb7-831a-822b209df03d None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:23.306357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-6deea611-7006-4cb7-831a-822b209df03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:23.340352 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508f7dd-864a-455e-a4ef-2c597e9cf557 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:23.371598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 463/959] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:23.372715 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deea611-7006-4cb7-831a-822b209df03d None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:23.382378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726a0f0-b295-405a-a919-650c116298d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:23.383331 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726a0f0-b295-405a-a919-650c116298d9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:23.383982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726a0f0-b295-405a-a919-650c116298d9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:23.383982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726a0f0-b295-405a-a919-650c116298d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:23.395295 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726a0f0-b295-405a-a919-650c116298d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:23.396775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726a0f0-b295-405a-a919-650c116298d9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-6726a0f0-b295-405a-a919-650c116298d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:23.413595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:23.414524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:23.415484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:23.415484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:23.415484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:23.415484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:23.419609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 497/960] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:23.420183 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726a0f0-b295-405a-a919-650c116298d9 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:23.471237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deea611-7006-4cb7-831a-822b209df03d None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:23.508944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d638b-8a16-42bd-99e6-c156e1dd479a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:23.509620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d638b-8a16-42bd-99e6-c156e1dd479a None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:23.509904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d638b-8a16-42bd-99e6-c156e1dd479a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:23.510180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d638b-8a16-42bd-99e6-c156e1dd479a None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:23.510887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d638b-8a16-42bd-99e6-c156e1dd479a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:23.511224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d638b-8a16-42bd-99e6-c156e1dd479a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:23.512581 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d638b-8a16-42bd-99e6-c156e1dd479a None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:23.517483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 464/961] 10.222.0.25 () {56 vars in 1122 bytes} [Mon May 9 18:17:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:23.518261 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d638b-8a16-42bd-99e6-c156e1dd479a None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:23.525610 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726a0f0-b295-405a-a919-650c116298d9 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:23.560299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1671d8-7196-4478-bf18-743a92d384da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:23.561024 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1671d8-7196-4478-bf18-743a92d384da None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:23.561304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1671d8-7196-4478-bf18-743a92d384da None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:23.561566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1671d8-7196-4478-bf18-743a92d384da None admin] PATH_INFO: `/v3/services/compute` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:23.562212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1671d8-7196-4478-bf18-743a92d384da None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:23.563630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1671d8-7196-4478-bf18-743a92d384da None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-ef1671d8-7196-4478-bf18-743a92d384da None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:23.575113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:23.576140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:23.576978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:23.576978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:23.576978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:23.576978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:23.576978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:23.576978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:23.576978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:23.576978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:23.576978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:23.576978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:23.576978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:23.576978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:23.576978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 18:17:23.576978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:23.581421 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 498/962] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:23.581996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1671d8-7196-4478-bf18-743a92d384da None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:23.615124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d638b-8a16-42bd-99e6-c156e1dd479a None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:23.649885 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7031a66-fc30-4dc9-8cff-6b1b97914b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:23.650550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7031a66-fc30-4dc9-8cff-6b1b97914b1e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:23.650831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7031a66-fc30-4dc9-8cff-6b1b97914b1e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:23.651104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7031a66-fc30-4dc9-8cff-6b1b97914b1e None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:23.651846 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7031a66-fc30-4dc9-8cff-6b1b97914b1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:23.652153 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7031a66-fc30-4dc9-8cff-6b1b97914b1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:23.653505 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7031a66-fc30-4dc9-8cff-6b1b97914b1e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:23.665295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 465/963] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:23.665843 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7031a66-fc30-4dc9-8cff-6b1b97914b1e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:23.678564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1671d8-7196-4478-bf18-743a92d384da None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:23.712537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850697a2-2c39-4ed2-a624-1e5f97ae397f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:23.713195 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850697a2-2c39-4ed2-a624-1e5f97ae397f None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:23.713474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850697a2-2c39-4ed2-a624-1e5f97ae397f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:23.713744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850697a2-2c39-4ed2-a624-1e5f97ae397f None admin] PATH_INFO: `/v3/endpoints` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:23.714459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850697a2-2c39-4ed2-a624-1e5f97ae397f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:23.715849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850697a2-2c39-4ed2-a624-1e5f97ae397f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:23.735620 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 499/964] 10.222.0.25 () {60 vars in 1144 bytes} [Mon May 9 18:17:23 2022] POST /identity/v3/endpoints => generated 349 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:17:23.736788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850697a2-2c39-4ed2-a624-1e5f97ae397f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:23.766051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7031a66-fc30-4dc9-8cff-6b1b97914b1e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:23.808566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f2f1b-918c-41dd-9cb2-d71370123d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:23.809261 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f2f1b-918c-41dd-9cb2-d71370123d43 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:23.809546 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f2f1b-918c-41dd-9cb2-d71370123d43 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:23.809823 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f2f1b-918c-41dd-9cb2-d71370123d43 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:23.810603 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f2f1b-918c-41dd-9cb2-d71370123d43 None admin] RBAC: Adding query filter params (name=compute) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:23.810858 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f2f1b-918c-41dd-9cb2-d71370123d43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:23.812314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f2f1b-918c-41dd-9cb2-d71370123d43 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:23.822388 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 466/965] 10.222.0.25 () {56 vars in 1112 bytes} [Mon May 9 18:17:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:23.823385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f2f1b-918c-41dd-9cb2-d71370123d43 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:23.880867 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850697a2-2c39-4ed2-a624-1e5f97ae397f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:23.921723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8f3fb-3b98-4725-bc11-970a7fc6d8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:23.922592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8f3fb-3b98-4725-bc11-970a7fc6d8c3 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:23.922874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8f3fb-3b98-4725-bc11-970a7fc6d8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:23.923145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8f3fb-3b98-4725-bc11-970a7fc6d8c3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:23.929634 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f2f1b-918c-41dd-9cb2-d71370123d43 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:23.931363 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8f3fb-3b98-4725-bc11-970a7fc6d8c3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:23.932832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8f3fb-3b98-4725-bc11-970a7fc6d8c3 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-0ba8f3fb-3b98-4725-bc11-970a7fc6d8c3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:17:23.948084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:23.949090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:23.950048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:23.950048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:23.950048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:17:23.950048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:17:23.950048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:17:23.950048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:17:23.950048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:17:23.950048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:17:23.950048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 18:17:23.950048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:23.955261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 500/966] 10.222.0.25 () {56 vars in 1117 bytes} [Mon May 9 18:17:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:23.956115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8f3fb-3b98-4725-bc11-970a7fc6d8c3 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:23.977160 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f739572-1732-4fdb-9122-8055dcf87c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:23.977160 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f739572-1732-4fdb-9122-8055dcf87c49 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:23.977415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f739572-1732-4fdb-9122-8055dcf87c49 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:23.977648 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f739572-1732-4fdb-9122-8055dcf87c49 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:23.978336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f739572-1732-4fdb-9122-8055dcf87c49 None admin] RBAC: Adding query filter params (type=compute) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:23.978643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f739572-1732-4fdb-9122-8055dcf87c49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:23.980079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f739572-1732-4fdb-9122-8055dcf87c49 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:23.986320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 467/967] 10.222.0.25 () {56 vars in 1112 bytes} [Mon May 9 18:17:23 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:24.071770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8f3fb-3b98-4725-bc11-970a7fc6d8c3 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:24.107848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5579c68-3d55-4ffb-a5fc-d6c4bdc1244b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:24.108832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5579c68-3d55-4ffb-a5fc-d6c4bdc1244b None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:24.109260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5579c68-3d55-4ffb-a5fc-d6c4bdc1244b None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:24.110233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5579c68-3d55-4ffb-a5fc-d6c4bdc1244b None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:24.111270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5579c68-3d55-4ffb-a5fc-d6c4bdc1244b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:24.111749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5579c68-3d55-4ffb-a5fc-d6c4bdc1244b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:24.113343 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5579c68-3d55-4ffb-a5fc-d6c4bdc1244b None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:24.134499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 501/968] 10.222.0.25 () {56 vars in 1149 bytes} [Mon May 9 18:17:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:24.139946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f739572-1732-4fdb-9122-8055dcf87c49 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:24.243114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f739572-1732-4fdb-9122-8055dcf87c49 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:24.276884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a4386-5a14-48c5-b132-aa4a254f4579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:24.277523 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a4386-5a14-48c5-b132-aa4a254f4579 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:24.277813 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a4386-5a14-48c5-b132-aa4a254f4579 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:24.278092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a4386-5a14-48c5-b132-aa4a254f4579 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:24.345935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a4386-5a14-48c5-b132-aa4a254f4579 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:24.347328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a4386-5a14-48c5-b132-aa4a254f4579 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:24.446515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 468/969] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:17:24 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:17:24.452281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5579c68-3d55-4ffb-a5fc-d6c4bdc1244b None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:24.552767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5579c68-3d55-4ffb-a5fc-d6c4bdc1244b None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:24.587276 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035b056-9737-449e-a131-99c32a0feae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:24.588076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035b056-9737-449e-a131-99c32a0feae6 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:24.588357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035b056-9737-449e-a131-99c32a0feae6 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:24.588630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035b056-9737-449e-a131-99c32a0feae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:24.601420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035b056-9737-449e-a131-99c32a0feae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:24.602802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035b056-9737-449e-a131-99c32a0feae6 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-8035b056-9737-449e-a131-99c32a0feae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:24.620515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:24.621472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:24.622396 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:24.622396 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:24.622396 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:24.622396 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:24.622396 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 502/970] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:24.627935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a4386-5a14-48c5-b132-aa4a254f4579 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:24.728772 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a4386-5a14-48c5-b132-aa4a254f4579 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:24.763842 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bb52f-6984-4500-af1a-cb885b2ba7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:24.764526 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bb52f-6984-4500-af1a-cb885b2ba7a1 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:24.764814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bb52f-6984-4500-af1a-cb885b2ba7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:24.765089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bb52f-6984-4500-af1a-cb885b2ba7a1 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:24.766345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bb52f-6984-4500-af1a-cb885b2ba7a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:24.766652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bb52f-6984-4500-af1a-cb885b2ba7a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:24.768052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bb52f-6984-4500-af1a-cb885b2ba7a1 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:24.779883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 469/971] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:24.786538 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035b056-9737-449e-a131-99c32a0feae6 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:24.887911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035b056-9737-449e-a131-99c32a0feae6 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:24.921918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c0c66-e318-4a0e-98e5-0ba109675e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:24.922741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c0c66-e318-4a0e-98e5-0ba109675e53 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:24.923021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c0c66-e318-4a0e-98e5-0ba109675e53 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:24.923326 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c0c66-e318-4a0e-98e5-0ba109675e53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:24.929935 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c0c66-e318-4a0e-98e5-0ba109675e53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:24.931400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c0c66-e318-4a0e-98e5-0ba109675e53 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-e65c0c66-e318-4a0e-98e5-0ba109675e53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:17:24.944839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:24.946923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:24.949237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:17:24.949237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:17:24.949237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:17:24.949237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:17:24.949237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:17:24.949237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:17:24.949237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:24.949237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:24.949237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 503/972] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:24.955199 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bb52f-6984-4500-af1a-cb885b2ba7a1 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:25.057320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bb52f-6984-4500-af1a-cb885b2ba7a1 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:25.094550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4303c67-1112-4678-9932-ddfb96feed4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:25.095274 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4303c67-1112-4678-9932-ddfb96feed4b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:25.095558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4303c67-1112-4678-9932-ddfb96feed4b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:25.095835 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4303c67-1112-4678-9932-ddfb96feed4b None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:25.096649 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4303c67-1112-4678-9932-ddfb96feed4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:25.096964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4303c67-1112-4678-9932-ddfb96feed4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:25.098444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4303c67-1112-4678-9932-ddfb96feed4b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:25.112404 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 470/973] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:17:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:25.117985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c0c66-e318-4a0e-98e5-0ba109675e53 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:25.220673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c0c66-e318-4a0e-98e5-0ba109675e53 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:25.254408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc4919-6b47-4e36-92f6-546c07cc26f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:25.255064 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc4919-6b47-4e36-92f6-546c07cc26f0 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:25.255385 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc4919-6b47-4e36-92f6-546c07cc26f0 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:25.255660 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc4919-6b47-4e36-92f6-546c07cc26f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:25.256536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc4919-6b47-4e36-92f6-546c07cc26f0 None admin] RBAC: Adding query filter params (role.id=104650f78a264c78a8cf5c6c4b389575, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=154f86b7575e47efb2a5e1c35943e037) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:25.256852 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc4919-6b47-4e36-92f6-546c07cc26f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:25.258254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc4919-6b47-4e36-92f6-546c07cc26f0 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:25.264425 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 504/974] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:17:25 2022] GET /identity/v3/role_assignments?user.id=154f86b7575e47efb2a5e1c35943e037&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=104650f78a264c78a8cf5c6c4b389575 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:25.277406 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4096a4-2997-42e2-b2a2-39626a49fb1b None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:25.277971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4096a4-2997-42e2-b2a2-39626a49fb1b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:25.277971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4096a4-2997-42e2-b2a2-39626a49fb1b None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:25.279688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 471/975] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:25.285689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91344d4e-7ea8-4d55-8e41-55950621672a None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:25.285987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91344d4e-7ea8-4d55-8e41-55950621672a None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:25.286251 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91344d4e-7ea8-4d55-8e41-55950621672a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:25.304294 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-91344d4e-7ea8-4d55-8e41-55950621672a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:25.305486 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-91344d4e-7ea8-4d55-8e41-55950621672a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:25.379241 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 505/976] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:25.403966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4096a4-2997-42e2-b2a2-39626a49fb1b None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:25.509428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4096a4-2997-42e2-b2a2-39626a49fb1b None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:25.543526 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c7cdf-40b1-4670-a545-0357795123fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:25.544580 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c7cdf-40b1-4670-a545-0357795123fe None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:25.545122 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c7cdf-40b1-4670-a545-0357795123fe None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:25.545604 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c7cdf-40b1-4670-a545-0357795123fe None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:25.546527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c7cdf-40b1-4670-a545-0357795123fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:25.548723 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c7cdf-40b1-4670-a545-0357795123fe None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:25.550251 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: WARNING keystone.server.flask.common [None req-a73c7cdf-40b1-4670-a545-0357795123fe None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 09 18:17:25.563747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.sql.core [None req-a73c7cdf-40b1-4670-a545-0357795123fe None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 18:17:25.563747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 18:17:25.563747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [parameters: {'user_id': '0bf2557a1925475baff2a05dddc62db9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 18:17:25.563747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79825) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-a73c7cdf-40b1-4670-a545-0357795123fe None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application result = self._query(query) May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application conn.query(q) May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application result.read() May 09 18:17:25.571981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 18:17:25.572844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application next(self.gen) May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application yield resource May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application next(self.gen) May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self.session.rollback() May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self.force_reraise() May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise self.value May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application session.commit() May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 18:17:25.573958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._prepare_impl() May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self.session.flush() May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._flush(objects) May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application compat.raise_( May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application flush_context.execute() May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rec.execute(self) May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 18:17:25.574899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application result = connection._execute_20( May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 18:17:25.575838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application result = self._query(query) May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application conn.query(q) May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application result.read() May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 18:17:25.576700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [parameters: {'user_id': '0bf2557a1925475baff2a05dddc62db9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:25.577548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 09 18:17:25.578421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 09 18:17:25.580924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 09 18:17:25.580924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 09 18:17:25.580924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 18:17:25.580924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 18:17:25.580924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 18:17:25.580924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:25.583575 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 472/977] 10.222.0.25 () {60 vars in 1131 bytes} [Mon May 9 18:17:25 2022] POST /identity/v3/users => generated 160 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:25.589676 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91344d4e-7ea8-4d55-8e41-55950621672a None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:25.691862 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91344d4e-7ea8-4d55-8e41-55950621672a None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:25.726729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21449472-155a-4e64-91ce-10232960e83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:25.728078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21449472-155a-4e64-91ce-10232960e83f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:25.728569 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21449472-155a-4e64-91ce-10232960e83f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:25.729047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21449472-155a-4e64-91ce-10232960e83f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:25.737566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21449472-155a-4e64-91ce-10232960e83f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:25.739354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21449472-155a-4e64-91ce-10232960e83f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-21449472-155a-4e64-91ce-10232960e83f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:17:25.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:25.755949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:25.756912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:25.756912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:25.756912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:17:25.756912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:17:25.756912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:17:25.756912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:17:25.756912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:17:25.756912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:17:25.756912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 18:17:25.756912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:25.759673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 506/978] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:17:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:25.765877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c7cdf-40b1-4670-a545-0357795123fe None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:25.869838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c7cdf-40b1-4670-a545-0357795123fe None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:25.904831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a4d50-3fc0-4807-920b-89984267ffdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:25.905864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a4d50-3fc0-4807-920b-89984267ffdc None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:25.906898 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a4d50-3fc0-4807-920b-89984267ffdc None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:25.907412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a4d50-3fc0-4807-920b-89984267ffdc None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:25.908426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a4d50-3fc0-4807-920b-89984267ffdc None admin] RBAC: Adding query filter params (name=cinder) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:25.909027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a4d50-3fc0-4807-920b-89984267ffdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:25.910856 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a4d50-3fc0-4807-920b-89984267ffdc None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:25.949696 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 473/979] 10.222.0.25 () {56 vars in 1101 bytes} [Mon May 9 18:17:25 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:26.185588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58921de2-1e7d-47e3-981b-4517f71431b7 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:26.185891 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58921de2-1e7d-47e3-981b-4517f71431b7 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:26.186074 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58921de2-1e7d-47e3-981b-4517f71431b7 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:26.188122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 507/980] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:26.193973 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3bb30-e5b8-46c3-8dcf-43b72510e333 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:26.194263 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3bb30-e5b8-46c3-8dcf-43b72510e333 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:26.194542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3bb30-e5b8-46c3-8dcf-43b72510e333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:26.213522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-aaf3bb30-e5b8-46c3-8dcf-43b72510e333 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:26.215026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-aaf3bb30-e5b8-46c3-8dcf-43b72510e333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:26.268727 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01277f1d-0d1a-428f-8f1d-51fb52baa995 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:26.269044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01277f1d-0d1a-428f-8f1d-51fb52baa995 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:26.269280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01277f1d-0d1a-428f-8f1d-51fb52baa995 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:26.270754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 508/981] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:26.275498 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8fe06-12db-4dce-8f6b-92be132e05bc None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:26.275786 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8fe06-12db-4dce-8f6b-92be132e05bc None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:26.276061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8fe06-12db-4dce-8f6b-92be132e05bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:26.283415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 474/982] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:26.292838 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-5eb8fe06-12db-4dce-8f6b-92be132e05bc None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:26.294001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-5eb8fe06-12db-4dce-8f6b-92be132e05bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:26.305115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3bb30-e5b8-46c3-8dcf-43b72510e333 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:26.359653 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 509/983] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:26.381897 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8fe06-12db-4dce-8f6b-92be132e05bc None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:26.400293 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3bb30-e5b8-46c3-8dcf-43b72510e333 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:26.434081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f48c9-415e-4bf0-b632-bf8c4486c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:26.434893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f48c9-415e-4bf0-b632-bf8c4486c2be None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:26.435212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f48c9-415e-4bf0-b632-bf8c4486c2be None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:26.435480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f48c9-415e-4bf0-b632-bf8c4486c2be None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:26.436195 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f48c9-415e-4bf0-b632-bf8c4486c2be None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:26.437497 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f48c9-415e-4bf0-b632-bf8c4486c2be None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-034f48c9-415e-4bf0-b632-bf8c4486c2be None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:26.449218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:26.450368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:26.456887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 475/984] 10.222.0.25 () {56 vars in 1126 bytes} [Mon May 9 18:17:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:26.457706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f48c9-415e-4bf0-b632-bf8c4486c2be None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:26.481037 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8fe06-12db-4dce-8f6b-92be132e05bc None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:26.515401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1067b-ada7-41b8-8559-c378165714ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:26.516632 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1067b-ada7-41b8-8559-c378165714ce None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:26.516913 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1067b-ada7-41b8-8559-c378165714ce None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:26.517190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1067b-ada7-41b8-8559-c378165714ce None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:26.517896 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1067b-ada7-41b8-8559-c378165714ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:26.519275 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1067b-ada7-41b8-8559-c378165714ce None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:26.533922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 510/985] 10.222.0.25 () {60 vars in 1141 bytes} [Mon May 9 18:17:26 2022] POST /identity/v3/services => generated 242 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:17:26.562364 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f48c9-415e-4bf0-b632-bf8c4486c2be None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:26.608184 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a845b52-fd82-4b24-a2bd-c580adf10986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:26.609173 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a845b52-fd82-4b24-a2bd-c580adf10986 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:26.609676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a845b52-fd82-4b24-a2bd-c580adf10986 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:26.610161 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a845b52-fd82-4b24-a2bd-c580adf10986 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:26.611132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a845b52-fd82-4b24-a2bd-c580adf10986 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:26.611663 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a845b52-fd82-4b24-a2bd-c580adf10986 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:26.613236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a845b52-fd82-4b24-a2bd-c580adf10986 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:26.619167 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 476/986] 10.222.0.25 () {56 vars in 1122 bytes} [Mon May 9 18:17:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:26.625986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1067b-ada7-41b8-8559-c378165714ce None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:26.726527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1067b-ada7-41b8-8559-c378165714ce None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:26.761601 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ce560-0d1a-49c9-8f91-65a625387155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:26.762271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ce560-0d1a-49c9-8f91-65a625387155 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:26.762555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ce560-0d1a-49c9-8f91-65a625387155 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:26.762834 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ce560-0d1a-49c9-8f91-65a625387155 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:26.763566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ce560-0d1a-49c9-8f91-65a625387155 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:26.763872 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ce560-0d1a-49c9-8f91-65a625387155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:26.765220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ce560-0d1a-49c9-8f91-65a625387155 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:26.770994 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 511/987] 10.222.0.25 () {56 vars in 1122 bytes} [Mon May 9 18:17:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:26.777044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a845b52-fd82-4b24-a2bd-c580adf10986 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:26.884313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a845b52-fd82-4b24-a2bd-c580adf10986 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:26.915624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324113c3-5992-4da4-9410-2df9192ee677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:26.916336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324113c3-5992-4da4-9410-2df9192ee677 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:26.916618 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324113c3-5992-4da4-9410-2df9192ee677 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:26.916891 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324113c3-5992-4da4-9410-2df9192ee677 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:26.917678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324113c3-5992-4da4-9410-2df9192ee677 None admin] RBAC: Adding query filter params (interface=admin, service_id=f3678b4b0d3841b39a5e3e7c59224083, region_id=RegionOne) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:26.917988 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324113c3-5992-4da4-9410-2df9192ee677 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:26.919356 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324113c3-5992-4da4-9410-2df9192ee677 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:26.929249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 477/988] 10.222.0.25 () {56 vars in 1249 bytes} [Mon May 9 18:17:26 2022] GET /identity/v3/endpoints?service_id=f3678b4b0d3841b39a5e3e7c59224083&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:26.930365 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ce560-0d1a-49c9-8f91-65a625387155 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:27.031768 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ce560-0d1a-49c9-8f91-65a625387155 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:27.066162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438a166-bbaf-4db2-8f8e-655740748de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:27.066834 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438a166-bbaf-4db2-8f8e-655740748de0 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:27.067112 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438a166-bbaf-4db2-8f8e-655740748de0 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:27.067410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438a166-bbaf-4db2-8f8e-655740748de0 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:27.068083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438a166-bbaf-4db2-8f8e-655740748de0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:27.069424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438a166-bbaf-4db2-8f8e-655740748de0 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:27.076010 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 512/989] 10.222.0.25 () {56 vars in 1087 bytes} [Mon May 9 18:17:26 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:17:27.591690 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d3a01-5cf5-43b7-9742-e5d8a92d9156 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:27.592205 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d3a01-5cf5-43b7-9742-e5d8a92d9156 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:27.592314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d3a01-5cf5-43b7-9742-e5d8a92d9156 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:27.593647 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 478/990] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:27.599427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c55db2-da11-4ab0-bf37-26a04c20873b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:27.599718 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c55db2-da11-4ab0-bf37-26a04c20873b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:27.599995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c55db2-da11-4ab0-bf37-26a04c20873b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:27.616661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c6c55db2-da11-4ab0-bf37-26a04c20873b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:27.617813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c6c55db2-da11-4ab0-bf37-26a04c20873b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:27.682774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 513/991] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:27 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:27.704381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d3a01-5cf5-43b7-9742-e5d8a92d9156 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:27.802926 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d3a01-5cf5-43b7-9742-e5d8a92d9156 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:27.835655 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fbb83-c32a-4de0-8575-504febac5e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:27.836452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fbb83-c32a-4de0-8575-504febac5e2a None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:27.836734 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fbb83-c32a-4de0-8575-504febac5e2a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:27.837010 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fbb83-c32a-4de0-8575-504febac5e2a None admin] PATH_INFO: `/v3/services/image` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:27.837651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fbb83-c32a-4de0-8575-504febac5e2a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:27.839023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fbb83-c32a-4de0-8575-504febac5e2a None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-397fbb83-c32a-4de0-8575-504febac5e2a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:27.851389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:27.852434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:27.853434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 479/992] 10.222.0.25 () {56 vars in 1105 bytes} [Mon May 9 18:17:27 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:27.858388 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c55db2-da11-4ab0-bf37-26a04c20873b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:27.951489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c55db2-da11-4ab0-bf37-26a04c20873b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:27.987936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83785c-8e0b-4c0a-a6f8-de52ec40d6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:27.988600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83785c-8e0b-4c0a-a6f8-de52ec40d6ad None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:27.988879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83785c-8e0b-4c0a-a6f8-de52ec40d6ad None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:27.989172 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83785c-8e0b-4c0a-a6f8-de52ec40d6ad None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:27.989864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83785c-8e0b-4c0a-a6f8-de52ec40d6ad None admin] RBAC: Adding query filter params (name=image) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:27.990165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83785c-8e0b-4c0a-a6f8-de52ec40d6ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:27.991539 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83785c-8e0b-4c0a-a6f8-de52ec40d6ad None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:28.001168 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 514/993] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:28.002593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fbb83-c32a-4de0-8575-504febac5e2a None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:28.098757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fbb83-c32a-4de0-8575-504febac5e2a None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:28.130824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f023b-dbcf-4d52-ba5d-d456ecd375e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:28.131439 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f023b-dbcf-4d52-ba5d-d456ecd375e0 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:28.131762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f023b-dbcf-4d52-ba5d-d456ecd375e0 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:28.132053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f023b-dbcf-4d52-ba5d-d456ecd375e0 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:28.132679 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f023b-dbcf-4d52-ba5d-d456ecd375e0 None admin] RBAC: Adding query filter params (type=image) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:28.132980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f023b-dbcf-4d52-ba5d-d456ecd375e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:28.134351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f023b-dbcf-4d52-ba5d-d456ecd375e0 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:28.138829 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 480/994] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:28 2022] GET /identity/v3/services?type=image => generated 122 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:28.782360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da38a1-9ab3-42b9-809f-219d0e0e8796 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:28.782890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da38a1-9ab3-42b9-809f-219d0e0e8796 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:28.783369 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da38a1-9ab3-42b9-809f-219d0e0e8796 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:28.785637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 515/995] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:28.791896 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46985657-d70b-49fb-9506-725c358e8f82 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:28.792359 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46985657-d70b-49fb-9506-725c358e8f82 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:28.792434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46985657-d70b-49fb-9506-725c358e8f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:28.810267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-46985657-d70b-49fb-9506-725c358e8f82 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:28.811479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-46985657-d70b-49fb-9506-725c358e8f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:28.877145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 481/996] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:28.898857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da38a1-9ab3-42b9-809f-219d0e0e8796 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:28.997431 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da38a1-9ab3-42b9-809f-219d0e0e8796 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:29.030732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772b252-60ee-4d26-adb4-7c4f5900e31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:29.031936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b252-60ee-4d26-adb4-7c4f5900e31e None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:29.032500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b252-60ee-4d26-adb4-7c4f5900e31e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:29.032980 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b252-60ee-4d26-adb4-7c4f5900e31e None admin] PATH_INFO: `/v3/services/compute` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:29.033895 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772b252-60ee-4d26-adb4-7c4f5900e31e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:29.035538 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772b252-60ee-4d26-adb4-7c4f5900e31e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-8772b252-60ee-4d26-adb4-7c4f5900e31e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:29.047255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:29.048336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:29.049212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:29.049212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:29.049212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:29.049212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:29.049212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:29.049212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:29.049212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:29.049212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:29.049212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:29.049212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:29.049212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:29.049212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:29.049212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 18:17:29.049212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:29.051321 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 516/997] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:29.055862 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46985657-d70b-49fb-9506-725c358e8f82 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:29.154101 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46985657-d70b-49fb-9506-725c358e8f82 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:29.186065 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2e78e-5892-43b5-b072-464ae2058018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:29.186731 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2e78e-5892-43b5-b072-464ae2058018 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:29.187014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2e78e-5892-43b5-b072-464ae2058018 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:29.187317 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2e78e-5892-43b5-b072-464ae2058018 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:29.188564 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2e78e-5892-43b5-b072-464ae2058018 None admin] RBAC: Adding query filter params (name=compute) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:29.188869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2e78e-5892-43b5-b072-464ae2058018 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:29.190216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2e78e-5892-43b5-b072-464ae2058018 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:29.199364 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 482/998] 10.222.0.25 () {56 vars in 1112 bytes} [Mon May 9 18:17:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:29.200909 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772b252-60ee-4d26-adb4-7c4f5900e31e None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:29.273635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17170aad-67a9-4583-b655-6eb6df7fdb5a None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:29.273961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17170aad-67a9-4583-b655-6eb6df7fdb5a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:29.274188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17170aad-67a9-4583-b655-6eb6df7fdb5a None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:29.275711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 483/999] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:17:29.280319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9244d99-30f9-4ee5-b5f8-0e5235e0b963 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:29.280602 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9244d99-30f9-4ee5-b5f8-0e5235e0b963 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:29.280878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9244d99-30f9-4ee5-b5f8-0e5235e0b963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:29.296701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-c9244d99-30f9-4ee5-b5f8-0e5235e0b963 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:29.298325 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c9244d99-30f9-4ee5-b5f8-0e5235e0b963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:29.299363 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772b252-60ee-4d26-adb4-7c4f5900e31e None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:29.335964 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ade4f-3e66-4b7d-8d36-216c3b2258ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:29.336631 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ade4f-3e66-4b7d-8d36-216c3b2258ba None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:29.336911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ade4f-3e66-4b7d-8d36-216c3b2258ba None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:29.337192 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ade4f-3e66-4b7d-8d36-216c3b2258ba None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:29.337886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ade4f-3e66-4b7d-8d36-216c3b2258ba None admin] RBAC: Adding query filter params (type=compute) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:29.338189 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ade4f-3e66-4b7d-8d36-216c3b2258ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:29.339576 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ade4f-3e66-4b7d-8d36-216c3b2258ba None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:29.344490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 517/1000] 10.222.0.25 () {56 vars in 1112 bytes} [Mon May 9 18:17:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:29.350029 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ade4f-3e66-4b7d-8d36-216c3b2258ba None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:29.379331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 484/1001] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:29.396498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9244d99-30f9-4ee5-b5f8-0e5235e0b963 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:29.446262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ade4f-3e66-4b7d-8d36-216c3b2258ba None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:29.478070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cd8fe-00cf-4f05-88df-fb046ceecdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:29.478715 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cd8fe-00cf-4f05-88df-fb046ceecdfb None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:29.478990 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cd8fe-00cf-4f05-88df-fb046ceecdfb None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:29.479354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cd8fe-00cf-4f05-88df-fb046ceecdfb None admin] PATH_INFO: `/v3/endpoints` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:29.480119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cd8fe-00cf-4f05-88df-fb046ceecdfb None admin] RBAC: Adding query filter params (interface=public, service_id=686a77dbeee64b819004c0fd67252e8f, region_id=RegionOne) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:29.480421 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cd8fe-00cf-4f05-88df-fb046ceecdfb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:29.481759 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cd8fe-00cf-4f05-88df-fb046ceecdfb None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:29.487195 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 518/1002] 10.222.0.25 () {56 vars in 1251 bytes} [Mon May 9 18:17:29 2022] GET /identity/v3/endpoints?service_id=686a77dbeee64b819004c0fd67252e8f&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:29.489903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9244d99-30f9-4ee5-b5f8-0e5235e0b963 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:29.492353 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cd8fe-00cf-4f05-88df-fb046ceecdfb None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:29.522379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7454e-6089-498f-89ed-5589d76e5d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:29.523302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7454e-6089-498f-89ed-5589d76e5d42 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:29.523593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7454e-6089-498f-89ed-5589d76e5d42 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:29.523866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7454e-6089-498f-89ed-5589d76e5d42 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:29.524502 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7454e-6089-498f-89ed-5589d76e5d42 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:29.525841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7454e-6089-498f-89ed-5589d76e5d42 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-89c7454e-6089-498f-89ed-5589d76e5d42 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:29.538229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:29.539248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:29.540137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 485/1003] 10.222.0.25 () {56 vars in 1126 bytes} [Mon May 9 18:17:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:29.545302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7454e-6089-498f-89ed-5589d76e5d42 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:29.590775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cd8fe-00cf-4f05-88df-fb046ceecdfb None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:29.622755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b3465-8f2f-43e2-b4b2-3d15cd3ab937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:29.623436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b3465-8f2f-43e2-b4b2-3d15cd3ab937 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:29.623716 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b3465-8f2f-43e2-b4b2-3d15cd3ab937 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:29.623985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b3465-8f2f-43e2-b4b2-3d15cd3ab937 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:29.624646 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b3465-8f2f-43e2-b4b2-3d15cd3ab937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:29.626019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b3465-8f2f-43e2-b4b2-3d15cd3ab937 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:29.632622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 519/1004] 10.222.0.25 () {56 vars in 1087 bytes} [Mon May 9 18:17:29 2022] GET /identity/v3/services => generated 1727 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:17:29.640833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7454e-6089-498f-89ed-5589d76e5d42 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:29.677875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea865f0-6c9b-48f8-bef4-959beea7d7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:29.678870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea865f0-6c9b-48f8-bef4-959beea7d7fb None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:29.679353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea865f0-6c9b-48f8-bef4-959beea7d7fb None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:29.679803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea865f0-6c9b-48f8-bef4-959beea7d7fb None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:29.680711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea865f0-6c9b-48f8-bef4-959beea7d7fb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:29.681191 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea865f0-6c9b-48f8-bef4-959beea7d7fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:29.682739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea865f0-6c9b-48f8-bef4-959beea7d7fb None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:29.688780 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 486/1005] 10.222.0.25 () {56 vars in 1122 bytes} [Mon May 9 18:17:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:29.694335 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b3465-8f2f-43e2-b4b2-3d15cd3ab937 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:29.797561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b3465-8f2f-43e2-b4b2-3d15cd3ab937 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:29.831570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf1f80-8d8d-4b9c-828c-937133b330e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:29.832242 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf1f80-8d8d-4b9c-828c-937133b330e8 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:29.832520 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf1f80-8d8d-4b9c-828c-937133b330e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:29.832791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf1f80-8d8d-4b9c-828c-937133b330e8 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:29.833484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf1f80-8d8d-4b9c-828c-937133b330e8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:29.833789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf1f80-8d8d-4b9c-828c-937133b330e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:29.835137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf1f80-8d8d-4b9c-828c-937133b330e8 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:29.840221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 520/1006] 10.222.0.25 () {56 vars in 1122 bytes} [Mon May 9 18:17:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:29.846347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea865f0-6c9b-48f8-bef4-959beea7d7fb None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:29.946383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea865f0-6c9b-48f8-bef4-959beea7d7fb None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:29.979820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb14ca-221c-4081-957a-ac6e5fb6c49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:29.980494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb14ca-221c-4081-957a-ac6e5fb6c49a None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:29.980774 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb14ca-221c-4081-957a-ac6e5fb6c49a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:29.981045 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb14ca-221c-4081-957a-ac6e5fb6c49a None admin] PATH_INFO: `/v3/endpoints` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:29.981791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb14ca-221c-4081-957a-ac6e5fb6c49a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:29.983145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb14ca-221c-4081-957a-ac6e5fb6c49a None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:29.998589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 487/1007] 10.222.0.25 () {60 vars in 1144 bytes} [Mon May 9 18:17:29 2022] POST /identity/v3/endpoints => generated 325 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:17:30.372447 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f4263-2346-4932-accc-26e2f144cc4d None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:30.372817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f4263-2346-4932-accc-26e2f144cc4d None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:30.373017 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f4263-2346-4932-accc-26e2f144cc4d None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:30.374510 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 521/1008] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:30.380035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344310f9-3cda-4a2a-81ce-73853c1b908c None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:30.380330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344310f9-3cda-4a2a-81ce-73853c1b908c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:30.380606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344310f9-3cda-4a2a-81ce-73853c1b908c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:30.397186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-344310f9-3cda-4a2a-81ce-73853c1b908c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:30.398350 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-344310f9-3cda-4a2a-81ce-73853c1b908c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:30.468041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 488/1009] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:30 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:30.491420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f4263-2346-4932-accc-26e2f144cc4d None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:30.590730 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f4263-2346-4932-accc-26e2f144cc4d None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:30.622006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f482d1-c157-42ff-a9c6-593c47ca4196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:30.622711 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f482d1-c157-42ff-a9c6-593c47ca4196 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:30.622938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f482d1-c157-42ff-a9c6-593c47ca4196 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:30.623233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f482d1-c157-42ff-a9c6-593c47ca4196 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:30.623946 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f482d1-c157-42ff-a9c6-593c47ca4196 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:30.625289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f482d1-c157-42ff-a9c6-593c47ca4196 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:30.639820 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 522/1010] 10.222.0.25 () {60 vars in 1141 bytes} [Mon May 9 18:17:30 2022] POST /identity/v3/services => generated 242 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:17:31.891317 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110283-4fb9-4906-a2b4-65767fdbc916 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:31.891835 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110283-4fb9-4906-a2b4-65767fdbc916 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:31.891949 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110283-4fb9-4906-a2b4-65767fdbc916 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:31.893372 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 489/1011] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:31.898376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cbc39-29d0-4bd4-8aca-dc48b87ab76e None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:31.898755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cbc39-29d0-4bd4-8aca-dc48b87ab76e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:31.898942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cbc39-29d0-4bd4-8aca-dc48b87ab76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:31.915288 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2d6cbc39-29d0-4bd4-8aca-dc48b87ab76e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:31.916447 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2d6cbc39-29d0-4bd4-8aca-dc48b87ab76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:31.990385 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 523/1012] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:31 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:32.014097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d110283-4fb9-4906-a2b4-65767fdbc916 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:32.123585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d110283-4fb9-4906-a2b4-65767fdbc916 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:32.156244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d23f5-2e5c-44cd-b01b-d378bdae4d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:32.156962 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d23f5-2e5c-44cd-b01b-d378bdae4d12 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:32.157242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d23f5-2e5c-44cd-b01b-d378bdae4d12 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:32.157514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d23f5-2e5c-44cd-b01b-d378bdae4d12 None admin] PATH_INFO: `/v3/services/compute` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:32.158153 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d23f5-2e5c-44cd-b01b-d378bdae4d12 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:32.159537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d23f5-2e5c-44cd-b01b-d378bdae4d12 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-9a7d23f5-2e5c-44cd-b01b-d378bdae4d12 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:32.170728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:32.172227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:32.173058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 490/1013] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:32.180015 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cbc39-29d0-4bd4-8aca-dc48b87ab76e None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:32.213589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a4b9-7d84-4507-9166-0b5fae95cfd0 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:32.213924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a4b9-7d84-4507-9166-0b5fae95cfd0 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:32.214133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a4b9-7d84-4507-9166-0b5fae95cfd0 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:32.215655 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 491/1014] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:17:32.220433 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d0d43-38fe-4505-adb9-d326d14ddac5 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:32.220719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d0d43-38fe-4505-adb9-d326d14ddac5 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:32.220992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d0d43-38fe-4505-adb9-d326d14ddac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:32.237607 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-e24d0d43-38fe-4505-adb9-d326d14ddac5 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:32.238981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e24d0d43-38fe-4505-adb9-d326d14ddac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:32.274043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cbc39-29d0-4bd4-8aca-dc48b87ab76e None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:32.306053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 492/1015] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:32 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:32.309005 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e7aac-5f1c-4593-ab20-a4bffe04e334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:32.309671 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e7aac-5f1c-4593-ab20-a4bffe04e334 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:32.309947 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e7aac-5f1c-4593-ab20-a4bffe04e334 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:32.310216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e7aac-5f1c-4593-ab20-a4bffe04e334 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:32.310904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e7aac-5f1c-4593-ab20-a4bffe04e334 None admin] RBAC: Adding query filter params (name=compute) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:32.311213 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e7aac-5f1c-4593-ab20-a4bffe04e334 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:32.313131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e7aac-5f1c-4593-ab20-a4bffe04e334 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:32.319313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 524/1016] 10.222.0.25 () {56 vars in 1112 bytes} [Mon May 9 18:17:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:32.323550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d0d43-38fe-4505-adb9-d326d14ddac5 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:32.329118 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e7aac-5f1c-4593-ab20-a4bffe04e334 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:32.418026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d0d43-38fe-4505-adb9-d326d14ddac5 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:32.424103 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e7aac-5f1c-4593-ab20-a4bffe04e334 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:32.451726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11ac94-c76d-4a2e-87e4-66073125e369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:32.452697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11ac94-c76d-4a2e-87e4-66073125e369 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:32.453002 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11ac94-c76d-4a2e-87e4-66073125e369 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:32.453277 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11ac94-c76d-4a2e-87e4-66073125e369 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:32.453971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11ac94-c76d-4a2e-87e4-66073125e369 None admin] RBAC: Adding query filter params (type=compute) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:32.454273 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11ac94-c76d-4a2e-87e4-66073125e369 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:32.455669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11ac94-c76d-4a2e-87e4-66073125e369 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:32.457766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea32779-61bc-49a9-9872-5d1bb0e0954c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:32.458558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea32779-61bc-49a9-9872-5d1bb0e0954c None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:32.458833 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea32779-61bc-49a9-9872-5d1bb0e0954c None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:32.459102 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea32779-61bc-49a9-9872-5d1bb0e0954c None admin] PATH_INFO: `/v3/domains/default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:32.460275 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea32779-61bc-49a9-9872-5d1bb0e0954c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:32.460751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 493/1017] 10.222.0.25 () {56 vars in 1112 bytes} [Mon May 9 18:17:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:32.464488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea32779-61bc-49a9-9872-5d1bb0e0954c None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:32.466102 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 525/1018] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:32.466910 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11ac94-c76d-4a2e-87e4-66073125e369 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:32.471967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea32779-61bc-49a9-9872-5d1bb0e0954c None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:32.565538 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea32779-61bc-49a9-9872-5d1bb0e0954c None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:32.576633 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11ac94-c76d-4a2e-87e4-66073125e369 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:32.601106 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf91b8-021c-47e1-a464-f02be91e4977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:32.601895 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf91b8-021c-47e1-a464-f02be91e4977 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:32.602093 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf91b8-021c-47e1-a464-f02be91e4977 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:32.602302 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf91b8-021c-47e1-a464-f02be91e4977 None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:32.602995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf91b8-021c-47e1-a464-f02be91e4977 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:32.604594 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf91b8-021c-47e1-a464-f02be91e4977 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:32.612243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5b23e-8154-4ec6-bb65-3154c1e8d8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:32.612907 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5b23e-8154-4ec6-bb65-3154c1e8d8ca None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:32.613182 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5b23e-8154-4ec6-bb65-3154c1e8d8ca None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:32.613456 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5b23e-8154-4ec6-bb65-3154c1e8d8ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:32.614180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5b23e-8154-4ec6-bb65-3154c1e8d8ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:32.615533 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5b23e-8154-4ec6-bb65-3154c1e8d8ca None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:32.631009 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 526/1019] 10.222.0.25 () {60 vars in 1141 bytes} [Mon May 9 18:17:32 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:17:32.635223 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 494/1020] 10.222.0.25 () {60 vars in 1144 bytes} [Mon May 9 18:17:32 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:17:32.894436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02d5c1-7b3a-4e50-aba0-9ef96f04aaf6 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:32.894714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02d5c1-7b3a-4e50-aba0-9ef96f04aaf6 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:32.894995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02d5c1-7b3a-4e50-aba0-9ef96f04aaf6 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:32.896404 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 527/1021] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:32.902292 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94c50e-8322-4c14-afd1-54d751cb9984 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:32.902753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94c50e-8322-4c14-afd1-54d751cb9984 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:32.902844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94c50e-8322-4c14-afd1-54d751cb9984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:32.920352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ec94c50e-8322-4c14-afd1-54d751cb9984 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:32.921529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ec94c50e-8322-4c14-afd1-54d751cb9984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:32.996011 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 495/1022] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:33.019454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02d5c1-7b3a-4e50-aba0-9ef96f04aaf6 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:33.117039 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02d5c1-7b3a-4e50-aba0-9ef96f04aaf6 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:33.149489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f9bb9-8ebf-4433-bb38-a3f9945e5419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:33.150200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f9bb9-8ebf-4433-bb38-a3f9945e5419 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:33.150474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f9bb9-8ebf-4433-bb38-a3f9945e5419 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:33.150738 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f9bb9-8ebf-4433-bb38-a3f9945e5419 None admin] PATH_INFO: `/v3/services/image` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:33.151399 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f9bb9-8ebf-4433-bb38-a3f9945e5419 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:33.152808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f9bb9-8ebf-4433-bb38-a3f9945e5419 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-e27f9bb9-8ebf-4433-bb38-a3f9945e5419 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:33.165131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:33.166144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:33.166991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 528/1023] 10.222.0.25 () {56 vars in 1105 bytes} [Mon May 9 18:17:33 2022] GET /identity/v3/services/image => generated 86 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:33.173846 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94c50e-8322-4c14-afd1-54d751cb9984 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:33.268858 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94c50e-8322-4c14-afd1-54d751cb9984 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:33.301513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86f498-c7b2-4f1a-8cd5-7192293d5809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:33.302175 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86f498-c7b2-4f1a-8cd5-7192293d5809 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:33.302454 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86f498-c7b2-4f1a-8cd5-7192293d5809 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:33.302721 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86f498-c7b2-4f1a-8cd5-7192293d5809 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:33.303506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb86f498-c7b2-4f1a-8cd5-7192293d5809 None admin] RBAC: Adding query filter params (name=image) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:33.303812 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb86f498-c7b2-4f1a-8cd5-7192293d5809 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:33.305146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb86f498-c7b2-4f1a-8cd5-7192293d5809 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:33.314841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 496/1024] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:33.316052 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f9bb9-8ebf-4433-bb38-a3f9945e5419 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:33.412622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f9bb9-8ebf-4433-bb38-a3f9945e5419 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:33.444047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ee625-792d-4812-ad87-e088bd1be162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:33.444713 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ee625-792d-4812-ad87-e088bd1be162 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:33.445034 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ee625-792d-4812-ad87-e088bd1be162 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:33.445310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ee625-792d-4812-ad87-e088bd1be162 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:33.445998 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ee625-792d-4812-ad87-e088bd1be162 None admin] RBAC: Adding query filter params (type=image) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:33.446296 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ee625-792d-4812-ad87-e088bd1be162 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:33.447686 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ee625-792d-4812-ad87-e088bd1be162 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:33.457906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 529/1025] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:33.459453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86f498-c7b2-4f1a-8cd5-7192293d5809 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:33.555100 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86f498-c7b2-4f1a-8cd5-7192293d5809 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:33.588668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3408be-cf11-477a-a14d-c540d39c5953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:33.589327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3408be-cf11-477a-a14d-c540d39c5953 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:33.589610 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3408be-cf11-477a-a14d-c540d39c5953 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:33.589888 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3408be-cf11-477a-a14d-c540d39c5953 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:33.590665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3408be-cf11-477a-a14d-c540d39c5953 None admin] RBAC: Adding query filter params (interface=public, service_id=cac6a5ec8dee423c9fdcda7188dbd681, region_id=RegionOne) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:33.590976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3408be-cf11-477a-a14d-c540d39c5953 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:33.592429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3408be-cf11-477a-a14d-c540d39c5953 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:33.602036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 497/1026] 10.222.0.25 () {56 vars in 1251 bytes} [Mon May 9 18:17:33 2022] GET /identity/v3/endpoints?service_id=cac6a5ec8dee423c9fdcda7188dbd681&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:33.603299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ee625-792d-4812-ad87-e088bd1be162 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:33.696877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ee625-792d-4812-ad87-e088bd1be162 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:33.727669 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28789b53-210f-4803-85b0-93204b2d8892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:33.728317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28789b53-210f-4803-85b0-93204b2d8892 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:33.728596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28789b53-210f-4803-85b0-93204b2d8892 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:33.728866 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28789b53-210f-4803-85b0-93204b2d8892 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:33.729527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28789b53-210f-4803-85b0-93204b2d8892 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:33.730871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28789b53-210f-4803-85b0-93204b2d8892 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:33.737953 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 530/1027] 10.222.0.25 () {56 vars in 1087 bytes} [Mon May 9 18:17:33 2022] GET /identity/v3/services => generated 1957 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:17:34.963150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a4cf4-4a11-4309-8bf8-bbdbb4445024 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:34.963570 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a4cf4-4a11-4309-8bf8-bbdbb4445024 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:34.963762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a4cf4-4a11-4309-8bf8-bbdbb4445024 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:34.965338 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 498/1028] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:34.972661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79234896-70a9-4e41-8e51-68adb200e799 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:34.973120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79234896-70a9-4e41-8e51-68adb200e799 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:34.973200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79234896-70a9-4e41-8e51-68adb200e799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:34.989356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-79234896-70a9-4e41-8e51-68adb200e799 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:34.990508 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-79234896-70a9-4e41-8e51-68adb200e799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:35.033316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459f31f-68c8-476d-b5f5-97b2ebfeee7c None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:35.033600 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459f31f-68c8-476d-b5f5-97b2ebfeee7c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:35.034076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459f31f-68c8-476d-b5f5-97b2ebfeee7c None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:35.035473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 499/1029] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:17:35.040464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b7c49-c3ee-429a-bc25-88fb3d4c84ad None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:35.040746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b7c49-c3ee-429a-bc25-88fb3d4c84ad None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:35.041018 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b7c49-c3ee-429a-bc25-88fb3d4c84ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:35.056192 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 531/1030] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:35.057331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-351b7c49-c3ee-429a-bc25-88fb3d4c84ad None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:35.058484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-351b7c49-c3ee-429a-bc25-88fb3d4c84ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:35.081161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79234896-70a9-4e41-8e51-68adb200e799 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:35.128637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 500/1031] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:35.151255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b7c49-c3ee-429a-bc25-88fb3d4c84ad None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:35.176173 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79234896-70a9-4e41-8e51-68adb200e799 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:35.216430 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b420f-30c6-4f82-abed-47376449face None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:35.217138 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b420f-30c6-4f82-abed-47376449face None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:35.217414 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b420f-30c6-4f82-abed-47376449face None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:35.217689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b420f-30c6-4f82-abed-47376449face None admin] PATH_INFO: `/v3/domains/default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:35.218873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b420f-30c6-4f82-abed-47376449face None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:35.220350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b420f-30c6-4f82-abed-47376449face None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:35.221941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 532/1032] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:35.227987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b420f-30c6-4f82-abed-47376449face None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:35.247111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b7c49-c3ee-429a-bc25-88fb3d4c84ad None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:35.280001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e72aa3-3fb4-41c2-bfec-2cb0f151b2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:35.280814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e72aa3-3fb4-41c2-bfec-2cb0f151b2ab None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:35.281146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e72aa3-3fb4-41c2-bfec-2cb0f151b2ab None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:35.281420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e72aa3-3fb4-41c2-bfec-2cb0f151b2ab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:35.291247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e72aa3-3fb4-41c2-bfec-2cb0f151b2ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-a8e72aa3-3fb4-41c2-bfec-2cb0f151b2ab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:35.299959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:35.300902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:35.301748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:35.301748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:35.301748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:35.301748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:35.301748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:35.301748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:35.301748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:35.301748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:35.301748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:35.301748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:35.301748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 18:17:35.301748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:35.301748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 501/1033] 10.222.0.25 () {56 vars in 1120 bytes} [Mon May 9 18:17:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:35.306896 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e72aa3-3fb4-41c2-bfec-2cb0f151b2ab None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:35.325409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b420f-30c6-4f82-abed-47376449face None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:35.357353 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b87fed-d930-4366-acc1-9ec9949748b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:35.358013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b87fed-d930-4366-acc1-9ec9949748b2 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:35.358289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b87fed-d930-4366-acc1-9ec9949748b2 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:35.358555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b87fed-d930-4366-acc1-9ec9949748b2 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:35.359789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b87fed-d930-4366-acc1-9ec9949748b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:35.361182 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b87fed-d930-4366-acc1-9ec9949748b2 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:35.383635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 533/1034] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:17:35 2022] POST /identity/v3/users => generated 285 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:17:35.406564 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e72aa3-3fb4-41c2-bfec-2cb0f151b2ab None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:35.442867 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b753a58-78bf-4d2f-ac70-c542aead0bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:35.443660 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753a58-78bf-4d2f-ac70-c542aead0bdf None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:35.443944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753a58-78bf-4d2f-ac70-c542aead0bdf None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:35.444305 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753a58-78bf-4d2f-ac70-c542aead0bdf None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:35.444976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b753a58-78bf-4d2f-ac70-c542aead0bdf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:35.445424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b753a58-78bf-4d2f-ac70-c542aead0bdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:35.446815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b753a58-78bf-4d2f-ac70-c542aead0bdf None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:35.456295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 502/1035] 10.222.0.25 () {56 vars in 1115 bytes} [Mon May 9 18:17:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:35.462474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b87fed-d930-4366-acc1-9ec9949748b2 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:35.567597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b87fed-d930-4366-acc1-9ec9949748b2 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:35.601692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d562c-7496-4cdb-b0dd-110a11c84b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:35.602629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d562c-7496-4cdb-b0dd-110a11c84b2a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:35.602812 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d562c-7496-4cdb-b0dd-110a11c84b2a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:35.603036 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d562c-7496-4cdb-b0dd-110a11c84b2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:35.614326 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d562c-7496-4cdb-b0dd-110a11c84b2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:35.615798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d562c-7496-4cdb-b0dd-110a11c84b2a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-6c0d562c-7496-4cdb-b0dd-110a11c84b2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:35.632787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:35.633929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:35.634877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:35.634877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:35.634877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:35.634877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:35.634877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 534/1036] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:35.639970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b753a58-78bf-4d2f-ac70-c542aead0bdf None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:35.737139 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b753a58-78bf-4d2f-ac70-c542aead0bdf None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:35.768522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4f668-f4bb-4223-b7eb-9af391c37b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:35.769246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4f668-f4bb-4223-b7eb-9af391c37b17 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:35.769534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4f668-f4bb-4223-b7eb-9af391c37b17 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:35.769807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4f668-f4bb-4223-b7eb-9af391c37b17 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:35.770518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4f668-f4bb-4223-b7eb-9af391c37b17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:35.770821 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4f668-f4bb-4223-b7eb-9af391c37b17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:35.772209 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4f668-f4bb-4223-b7eb-9af391c37b17 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:35.784739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 503/1037] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:35.790333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d562c-7496-4cdb-b0dd-110a11c84b2a None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:35.888406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d562c-7496-4cdb-b0dd-110a11c84b2a None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:35.920662 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d2d69-0cfa-4766-a490-e37a8883d3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:35.921474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d2d69-0cfa-4766-a490-e37a8883d3bd None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:35.921757 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d2d69-0cfa-4766-a490-e37a8883d3bd None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:35.922030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d2d69-0cfa-4766-a490-e37a8883d3bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:35.929301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d2d69-0cfa-4766-a490-e37a8883d3bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:35.930736 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d2d69-0cfa-4766-a490-e37a8883d3bd None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-8f9d2d69-0cfa-4766-a490-e37a8883d3bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:17:35.944434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:35.945408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:35.946373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:35.946373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:35.946373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:17:35.946373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:17:35.946373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:17:35.946373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:17:35.946373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:17:35.946373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:17:35.946373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 18:17:35.946373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:35.950349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 535/1038] 10.222.0.25 () {56 vars in 1093 bytes} [Mon May 9 18:17:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:35.951376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4f668-f4bb-4223-b7eb-9af391c37b17 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:35.980700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33572095-9abd-449f-aacc-556f4edcd646 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:35.981294 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33572095-9abd-449f-aacc-556f4edcd646 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:35.981294 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33572095-9abd-449f-aacc-556f4edcd646 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:35.982723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 536/1039] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:35.987818 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5578b90-db22-4c7e-a521-52df928cf993 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:35.988107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5578b90-db22-4c7e-a521-52df928cf993 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:35.988393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5578b90-db22-4c7e-a521-52df928cf993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:36.005130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-d5578b90-db22-4c7e-a521-52df928cf993 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:36.006292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-d5578b90-db22-4c7e-a521-52df928cf993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:36.051556 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4f668-f4bb-4223-b7eb-9af391c37b17 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:36.073957 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 537/1040] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:36.086939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb86eb-04fb-4e03-a7f8-c7c6db429be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:36.087711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb86eb-04fb-4e03-a7f8-c7c6db429be4 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:36.088064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb86eb-04fb-4e03-a7f8-c7c6db429be4 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:36.088409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb86eb-04fb-4e03-a7f8-c7c6db429be4 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:36.089302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb86eb-04fb-4e03-a7f8-c7c6db429be4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:36.089677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb86eb-04fb-4e03-a7f8-c7c6db429be4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:36.091198 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb86eb-04fb-4e03-a7f8-c7c6db429be4 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:36.097330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5578b90-db22-4c7e-a521-52df928cf993 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:36.108795 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 504/1041] 10.222.0.25 () {56 vars in 1133 bytes} [Mon May 9 18:17:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:36.113999 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb86eb-04fb-4e03-a7f8-c7c6db429be4 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:36.199387 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5578b90-db22-4c7e-a521-52df928cf993 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:36.212826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb86eb-04fb-4e03-a7f8-c7c6db429be4 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:36.234330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdf7a0-17ad-48d7-8903-a7d6f90fafde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:36.235108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdf7a0-17ad-48d7-8903-a7d6f90fafde None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:36.235411 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdf7a0-17ad-48d7-8903-a7d6f90fafde None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:36.235677 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdf7a0-17ad-48d7-8903-a7d6f90fafde None admin] PATH_INFO: `/v3/services/image` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:36.236318 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdf7a0-17ad-48d7-8903-a7d6f90fafde None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:36.237656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdf7a0-17ad-48d7-8903-a7d6f90fafde None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:36.245900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8e6aa-ff6c-404a-8211-694278e3021a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:36.246547 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8e6aa-ff6c-404a-8211-694278e3021a None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:36.246828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8e6aa-ff6c-404a-8211-694278e3021a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:36.247102 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8e6aa-ff6c-404a-8211-694278e3021a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-b2bdf7a0-17ad-48d7-8903-a7d6f90fafde None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:36.248816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:36.250270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:36.251105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:36.251105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:36.251105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:36.251105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:36.251105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:36.251105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:36.251105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:36.251105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:36.251105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:36.251105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:36.251105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:17:36.251105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:17:36.251105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 18:17:36.251105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:36.255341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 538/1042] 10.222.0.25 () {56 vars in 1105 bytes} [Mon May 9 18:17:36 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:36.256585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdf7a0-17ad-48d7-8903-a7d6f90fafde None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:36.319670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea8e6aa-ff6c-404a-8211-694278e3021a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:36.321103 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea8e6aa-ff6c-404a-8211-694278e3021a None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:36.355068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdf7a0-17ad-48d7-8903-a7d6f90fafde None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:36.389299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5710171-2eee-41df-b2e4-165edc23ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:36.389948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5710171-2eee-41df-b2e4-165edc23ea1b None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:36.390262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5710171-2eee-41df-b2e4-165edc23ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:36.390537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5710171-2eee-41df-b2e4-165edc23ea1b None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:36.391249 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5710171-2eee-41df-b2e4-165edc23ea1b None admin] RBAC: Adding query filter params (name=image) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:36.391559 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5710171-2eee-41df-b2e4-165edc23ea1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:36.392902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5710171-2eee-41df-b2e4-165edc23ea1b None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:36.402234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 539/1043] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:36.402870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5710171-2eee-41df-b2e4-165edc23ea1b None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:36.416156 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 505/1044] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:17:36 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 18:17:36.421345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8e6aa-ff6c-404a-8211-694278e3021a None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:36.505416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5710171-2eee-41df-b2e4-165edc23ea1b None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:36.519163 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8e6aa-ff6c-404a-8211-694278e3021a None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:36.539271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d9822-e2a2-45fa-b301-1aaf28e74516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:36.539934 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d9822-e2a2-45fa-b301-1aaf28e74516 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:36.540216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d9822-e2a2-45fa-b301-1aaf28e74516 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:36.540493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d9822-e2a2-45fa-b301-1aaf28e74516 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:36.541187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d9822-e2a2-45fa-b301-1aaf28e74516 None admin] RBAC: Adding query filter params (type=image) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:36.541492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d9822-e2a2-45fa-b301-1aaf28e74516 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:36.542841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d9822-e2a2-45fa-b301-1aaf28e74516 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:36.548414 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 540/1045] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:36.554702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d9822-e2a2-45fa-b301-1aaf28e74516 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:36.555304 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44485c84-6f5f-4ff6-8cc9-a742779e8f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:36.556114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44485c84-6f5f-4ff6-8cc9-a742779e8f6f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:36.556399 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44485c84-6f5f-4ff6-8cc9-a742779e8f6f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:36.556679 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44485c84-6f5f-4ff6-8cc9-a742779e8f6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:36.567444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44485c84-6f5f-4ff6-8cc9-a742779e8f6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:36.568940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44485c84-6f5f-4ff6-8cc9-a742779e8f6f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-44485c84-6f5f-4ff6-8cc9-a742779e8f6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:36.594698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:36.595701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:36.596597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:36.596597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:36.596597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:36.596597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:36.597407 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 506/1046] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:36.601375 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44485c84-6f5f-4ff6-8cc9-a742779e8f6f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:36.660533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d9822-e2a2-45fa-b301-1aaf28e74516 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:36.692265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e2bd1-b4cc-43c5-a2b7-196f3b1e5a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:36.692918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e2bd1-b4cc-43c5-a2b7-196f3b1e5a0a None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:36.693197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e2bd1-b4cc-43c5-a2b7-196f3b1e5a0a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:36.693464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e2bd1-b4cc-43c5-a2b7-196f3b1e5a0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:36.694188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e2bd1-b4cc-43c5-a2b7-196f3b1e5a0a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:36.695558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e2bd1-b4cc-43c5-a2b7-196f3b1e5a0a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:36.697234 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44485c84-6f5f-4ff6-8cc9-a742779e8f6f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:36.723406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 541/1047] 10.222.0.25 () {60 vars in 1144 bytes} [Mon May 9 18:17:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:17:36.738759 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2f11c-e78e-4c7b-8b21-1981ab98fab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:36.739479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2f11c-e78e-4c7b-8b21-1981ab98fab1 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:36.739760 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2f11c-e78e-4c7b-8b21-1981ab98fab1 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:36.740031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2f11c-e78e-4c7b-8b21-1981ab98fab1 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:36.740764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2f11c-e78e-4c7b-8b21-1981ab98fab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:36.741073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2f11c-e78e-4c7b-8b21-1981ab98fab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:36.742496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2f11c-e78e-4c7b-8b21-1981ab98fab1 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:36.760458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 507/1048] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:36.762042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e2bd1-b4cc-43c5-a2b7-196f3b1e5a0a None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:36.868326 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e2bd1-b4cc-43c5-a2b7-196f3b1e5a0a None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:36.902099 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20695d2a-9df0-47fe-94d3-27e02deb81d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:36.902894 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20695d2a-9df0-47fe-94d3-27e02deb81d9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:36.903204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20695d2a-9df0-47fe-94d3-27e02deb81d9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:36.903482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20695d2a-9df0-47fe-94d3-27e02deb81d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:36.914885 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20695d2a-9df0-47fe-94d3-27e02deb81d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:36.917616 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20695d2a-9df0-47fe-94d3-27e02deb81d9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-20695d2a-9df0-47fe-94d3-27e02deb81d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:17:36.935531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:36.936599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:36.937700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:17:36.937700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:17:36.937700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:17:36.937700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:17:36.937700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:17:36.937700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:17:36.937700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:36.937700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:36.937700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 542/1049] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:36.943904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2f11c-e78e-4c7b-8b21-1981ab98fab1 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:37.043020 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2f11c-e78e-4c7b-8b21-1981ab98fab1 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:37.075579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945dfe33-0058-4553-8cd2-2aa9353e6fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:37.076297 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945dfe33-0058-4553-8cd2-2aa9353e6fe6 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:37.076527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945dfe33-0058-4553-8cd2-2aa9353e6fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:37.076784 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945dfe33-0058-4553-8cd2-2aa9353e6fe6 None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:37.077568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945dfe33-0058-4553-8cd2-2aa9353e6fe6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:37.078484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945dfe33-0058-4553-8cd2-2aa9353e6fe6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:37.080077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945dfe33-0058-4553-8cd2-2aa9353e6fe6 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:37.092184 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 508/1050] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:17:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:37.098192 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20695d2a-9df0-47fe-94d3-27e02deb81d9 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:37.201501 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20695d2a-9df0-47fe-94d3-27e02deb81d9 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:37.236391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc5e28-dff8-4f5e-9944-7304f771b57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:37.237043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc5e28-dff8-4f5e-9944-7304f771b57f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:37.237322 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc5e28-dff8-4f5e-9944-7304f771b57f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:37.237595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc5e28-dff8-4f5e-9944-7304f771b57f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:37.238415 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc5e28-dff8-4f5e-9944-7304f771b57f None admin] RBAC: Adding query filter params (role.id=104650f78a264c78a8cf5c6c4b389575, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=f769ee9f31354e5296645fb1e3bd84b6) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:37.238720 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc5e28-dff8-4f5e-9944-7304f771b57f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:37.240159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc5e28-dff8-4f5e-9944-7304f771b57f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:37.245812 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 543/1051] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:17:37 2022] GET /identity/v3/role_assignments?user.id=f769ee9f31354e5296645fb1e3bd84b6&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=104650f78a264c78a8cf5c6c4b389575 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:37.708646 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d00d40-9a01-41ed-95c1-d7dae9eae379 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:37.708967 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d00d40-9a01-41ed-95c1-d7dae9eae379 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:37.709207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d00d40-9a01-41ed-95c1-d7dae9eae379 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:37.710609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 509/1052] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:37.715879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb92f8a-ab42-4a09-9570-4ba1f461dda6 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:37.716169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb92f8a-ab42-4a09-9570-4ba1f461dda6 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:37.716447 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb92f8a-ab42-4a09-9570-4ba1f461dda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:37.733468 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-7bb92f8a-ab42-4a09-9570-4ba1f461dda6 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:37.735302 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7bb92f8a-ab42-4a09-9570-4ba1f461dda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:37.803270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 544/1053] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:37.826123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d00d40-9a01-41ed-95c1-d7dae9eae379 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:37.925993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d00d40-9a01-41ed-95c1-d7dae9eae379 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:37.959059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567e26-8efd-4b75-9bb4-e55cc138657f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:37.959822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567e26-8efd-4b75-9bb4-e55cc138657f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:37.960107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567e26-8efd-4b75-9bb4-e55cc138657f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:37.960381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567e26-8efd-4b75-9bb4-e55cc138657f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:37.973667 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26567e26-8efd-4b75-9bb4-e55cc138657f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-26567e26-8efd-4b75-9bb4-e55cc138657f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:37.981318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:37.982283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:37.983149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:37.983149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:37.983149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:37.983149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:37.983149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:37.983149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:37.983149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:37.983149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:37.983149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:37.983149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:37.983149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:17:37.983149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:37.983149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 510/1054] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:17:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:37.990858 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb92f8a-ab42-4a09-9570-4ba1f461dda6 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:38.085818 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb92f8a-ab42-4a09-9570-4ba1f461dda6 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:38.119367 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c0b9e-d34d-437b-9c56-f98c26b12dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:38.120107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c0b9e-d34d-437b-9c56-f98c26b12dde None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:38.120292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c0b9e-d34d-437b-9c56-f98c26b12dde None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:38.120568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c0b9e-d34d-437b-9c56-f98c26b12dde None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:38.121295 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c0b9e-d34d-437b-9c56-f98c26b12dde None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:38.121594 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c0b9e-d34d-437b-9c56-f98c26b12dde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:38.122936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c0b9e-d34d-437b-9c56-f98c26b12dde None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:38.135447 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 545/1055] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:17:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:38.136624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567e26-8efd-4b75-9bb4-e55cc138657f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:38.231286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567e26-8efd-4b75-9bb4-e55cc138657f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:38.263007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c6d20-38d2-49e1-aed5-ded0dfb2fe58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:38.264201 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c6d20-38d2-49e1-aed5-ded0dfb2fe58 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:38.264630 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c6d20-38d2-49e1-aed5-ded0dfb2fe58 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:38.265558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c6d20-38d2-49e1-aed5-ded0dfb2fe58 None admin] PATH_INFO: `/v3/users/99bbf92cdba043a094585191d88d9ebf` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:38.278380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c6d20-38d2-49e1-aed5-ded0dfb2fe58 None admin] RBAC: Authorizing `identity:get_user(user_id=99bbf92cdba043a094585191d88d9ebf)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:38.280026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c6d20-38d2-49e1-aed5-ded0dfb2fe58 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:38.286850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 511/1056] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:17:38 2022] GET /identity/v3/users/99bbf92cdba043a094585191d88d9ebf => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:38.288214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c0b9e-d34d-437b-9c56-f98c26b12dde None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:38.394118 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c0b9e-d34d-437b-9c56-f98c26b12dde None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:38.426219 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286ed76-1c07-4252-a240-87ceb3053216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:38.427125 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286ed76-1c07-4252-a240-87ceb3053216 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:38.427587 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286ed76-1c07-4252-a240-87ceb3053216 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:38.428026 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286ed76-1c07-4252-a240-87ceb3053216 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:38.492288 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e286ed76-1c07-4252-a240-87ceb3053216 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:38.493937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e286ed76-1c07-4252-a240-87ceb3053216 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:38.592474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 546/1057] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:17:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:17:38.593807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c6d20-38d2-49e1-aed5-ded0dfb2fe58 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:38.686665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c6d20-38d2-49e1-aed5-ded0dfb2fe58 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:38.719991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8def65-c5c1-415c-af09-e30cdaa0616e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:38.720846 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8def65-c5c1-415c-af09-e30cdaa0616e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:38.721132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8def65-c5c1-415c-af09-e30cdaa0616e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:38.721405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8def65-c5c1-415c-af09-e30cdaa0616e None admin] PATH_INFO: `/v3/projects/cd6a6211651144beb028e5b0f088028f` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:38.722562 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8def65-c5c1-415c-af09-e30cdaa0616e None admin] RBAC: Authorizing `identity:get_project(project_id=cd6a6211651144beb028e5b0f088028f)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:38.724005 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8def65-c5c1-415c-af09-e30cdaa0616e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:38.730771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 512/1058] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:17:38 2022] GET /identity/v3/projects/cd6a6211651144beb028e5b0f088028f => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:38.731751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286ed76-1c07-4252-a240-87ceb3053216 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:38.829904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286ed76-1c07-4252-a240-87ceb3053216 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:38.863480 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d94d7-cc6c-4953-a416-408838a3f3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:38.864216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d94d7-cc6c-4953-a416-408838a3f3f8 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:38.864497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d94d7-cc6c-4953-a416-408838a3f3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:38.864818 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d94d7-cc6c-4953-a416-408838a3f3f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:38.865634 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d94d7-cc6c-4953-a416-408838a3f3f8 None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=cd6a6211651144beb028e5b0f088028f, user.id=99bbf92cdba043a094585191d88d9ebf) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:38.865939 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d94d7-cc6c-4953-a416-408838a3f3f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:38.867363 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d94d7-cc6c-4953-a416-408838a3f3f8 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:38.879225 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 547/1059] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:17:38 2022] GET /identity/v3/role_assignments?user.id=99bbf92cdba043a094585191d88d9ebf&scope.project.id=cd6a6211651144beb028e5b0f088028f&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:38.992351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f32bd-b0b7-4069-b71b-ecaa8c5d5756 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:38.992670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f32bd-b0b7-4069-b71b-ecaa8c5d5756 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:38.992898 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f32bd-b0b7-4069-b71b-ecaa8c5d5756 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:38.994463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 513/1060] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:39.000710 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed515373-733c-45fe-8f05-59586bc03782 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:39.001014 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed515373-733c-45fe-8f05-59586bc03782 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:39.001259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed515373-733c-45fe-8f05-59586bc03782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:39.018860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-ed515373-733c-45fe-8f05-59586bc03782 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:39.020073 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ed515373-733c-45fe-8f05-59586bc03782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:39.086376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 548/1061] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:39.109339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f32bd-b0b7-4069-b71b-ecaa8c5d5756 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:39.207036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f32bd-b0b7-4069-b71b-ecaa8c5d5756 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:39.239550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94542326-000c-4ee3-9c19-921ebb6eb949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:39.240198 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94542326-000c-4ee3-9c19-921ebb6eb949 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:39.240475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94542326-000c-4ee3-9c19-921ebb6eb949 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:39.240748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94542326-000c-4ee3-9c19-921ebb6eb949 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:39.309501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94542326-000c-4ee3-9c19-921ebb6eb949 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:39.310929 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94542326-000c-4ee3-9c19-921ebb6eb949 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:39.411062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 514/1062] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:17:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:17:39.412899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed515373-733c-45fe-8f05-59586bc03782 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:39.446165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7e34-6758-49cc-a170-69370c3dbf10 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:39.446472 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7e34-6758-49cc-a170-69370c3dbf10 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:39.446733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7e34-6758-49cc-a170-69370c3dbf10 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:39.448275 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 515/1063] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:17:39.453199 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb31d5-64db-4a65-aeff-b7e73a2796ce None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:39.453489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb31d5-64db-4a65-aeff-b7e73a2796ce None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:39.453770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb31d5-64db-4a65-aeff-b7e73a2796ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:39.470798 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-71bb31d5-64db-4a65-aeff-b7e73a2796ce None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:39.472000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-71bb31d5-64db-4a65-aeff-b7e73a2796ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:39.513190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed515373-733c-45fe-8f05-59586bc03782 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:39.537574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 516/1064] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:39.548026 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94012911-f11f-4aec-933b-c8bd9ce32dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:39.548813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94012911-f11f-4aec-933b-c8bd9ce32dd3 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:39.549176 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94012911-f11f-4aec-933b-c8bd9ce32dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:39.549350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94012911-f11f-4aec-933b-c8bd9ce32dd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:39.560405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94012911-f11f-4aec-933b-c8bd9ce32dd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:39.561837 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94012911-f11f-4aec-933b-c8bd9ce32dd3 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:39.563169 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb31d5-64db-4a65-aeff-b7e73a2796ce None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-94012911-f11f-4aec-933b-c8bd9ce32dd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:39.579300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:39.580236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:39.581210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:39.581210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:39.581210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:39.581210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:39.581210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 549/1065] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:39.586073 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94012911-f11f-4aec-933b-c8bd9ce32dd3 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:39.661828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb31d5-64db-4a65-aeff-b7e73a2796ce None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:39.682388 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94012911-f11f-4aec-933b-c8bd9ce32dd3 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:39.696805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f033b09-8fc4-4b30-8d59-e149e42a2e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:39.697626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f033b09-8fc4-4b30-8d59-e149e42a2e69 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:39.697904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f033b09-8fc4-4b30-8d59-e149e42a2e69 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:39.698180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f033b09-8fc4-4b30-8d59-e149e42a2e69 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:39.708780 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f033b09-8fc4-4b30-8d59-e149e42a2e69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:39.715029 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bac41e-9a10-48e2-8ca1-c776cd9a55ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:39.715715 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bac41e-9a10-48e2-8ca1-c776cd9a55ca None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:39.715985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bac41e-9a10-48e2-8ca1-c776cd9a55ca None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:39.716259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bac41e-9a10-48e2-8ca1-c776cd9a55ca None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:39.716963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bac41e-9a10-48e2-8ca1-c776cd9a55ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-1f033b09-8fc4-4b30-8d59-e149e42a2e69 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:39.717243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:39.718788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:39.724264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:39.724264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:39.724264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:39.724264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:39.724264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:39.724264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:39.724264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:39.724264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:39.724264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:39.724264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:39.724264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 18:17:39.724264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:39.724931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bac41e-9a10-48e2-8ca1-c776cd9a55ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:39.724931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bac41e-9a10-48e2-8ca1-c776cd9a55ca None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:39.725068 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 517/1066] 10.222.0.25 () {56 vars in 1120 bytes} [Mon May 9 18:17:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:39.726092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f033b09-8fc4-4b30-8d59-e149e42a2e69 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:39.730291 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 550/1067] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:39.833756 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f033b09-8fc4-4b30-8d59-e149e42a2e69 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:39.867685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0bd80-e560-47c4-82b1-655ba37dab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:39.868381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0bd80-e560-47c4-82b1-655ba37dab0c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:39.868666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0bd80-e560-47c4-82b1-655ba37dab0c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:39.868944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0bd80-e560-47c4-82b1-655ba37dab0c None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:39.869704 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0bd80-e560-47c4-82b1-655ba37dab0c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:39.870013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0bd80-e560-47c4-82b1-655ba37dab0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:39.871421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0bd80-e560-47c4-82b1-655ba37dab0c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:39.885049 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 518/1068] 10.222.0.25 () {56 vars in 1115 bytes} [Mon May 9 18:17:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:39.886423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bac41e-9a10-48e2-8ca1-c776cd9a55ca None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:39.988984 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bac41e-9a10-48e2-8ca1-c776cd9a55ca None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:40.023672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9416a8d-39c4-48d9-b9c7-2f57122df9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:40.024472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9416a8d-39c4-48d9-b9c7-2f57122df9ac None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:40.024758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9416a8d-39c4-48d9-b9c7-2f57122df9ac None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:40.025033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9416a8d-39c4-48d9-b9c7-2f57122df9ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:40.035839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9416a8d-39c4-48d9-b9c7-2f57122df9ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:40.037288 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9416a8d-39c4-48d9-b9c7-2f57122df9ac None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-b9416a8d-39c4-48d9-b9c7-2f57122df9ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:40.054709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:40.055690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:40.056592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:40.056592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:40.056592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:40.056592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:40.056592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 551/1069] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:40.061730 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0bd80-e560-47c4-82b1-655ba37dab0c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:40.157669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0bd80-e560-47c4-82b1-655ba37dab0c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:40.189354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820ba5e-3721-4e6f-a8ec-1549814c0b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:40.190011 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820ba5e-3721-4e6f-a8ec-1549814c0b01 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:40.190295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820ba5e-3721-4e6f-a8ec-1549814c0b01 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:40.190573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820ba5e-3721-4e6f-a8ec-1549814c0b01 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:40.191460 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820ba5e-3721-4e6f-a8ec-1549814c0b01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:40.191772 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820ba5e-3721-4e6f-a8ec-1549814c0b01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:40.193141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820ba5e-3721-4e6f-a8ec-1549814c0b01 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:40.204798 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 519/1070] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:40.210462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9416a8d-39c4-48d9-b9c7-2f57122df9ac None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:40.303550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9416a8d-39c4-48d9-b9c7-2f57122df9ac None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:40.335046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb748f-f4c7-4fcd-a189-e16d49dc6e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:40.335861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb748f-f4c7-4fcd-a189-e16d49dc6e12 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:40.336140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb748f-f4c7-4fcd-a189-e16d49dc6e12 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:40.336410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb748f-f4c7-4fcd-a189-e16d49dc6e12 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:40.343485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb748f-f4c7-4fcd-a189-e16d49dc6e12 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:40.344890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb748f-f4c7-4fcd-a189-e16d49dc6e12 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-cccb748f-f4c7-4fcd-a189-e16d49dc6e12 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:17:40.358149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:40.359638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:40.361200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:40.361200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:40.361200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:17:40.361200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:17:40.361200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:17:40.361200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:17:40.361200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:17:40.361200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:17:40.361200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 18:17:40.361200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:40.361200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 552/1071] 10.222.0.25 () {56 vars in 1093 bytes} [Mon May 9 18:17:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:40.365564 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820ba5e-3721-4e6f-a8ec-1549814c0b01 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:40.460052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820ba5e-3721-4e6f-a8ec-1549814c0b01 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:40.491896 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c700b36-2b1f-4045-a8c3-73726a180957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:40.492563 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c700b36-2b1f-4045-a8c3-73726a180957 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:40.492846 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c700b36-2b1f-4045-a8c3-73726a180957 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:40.493121 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c700b36-2b1f-4045-a8c3-73726a180957 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:40.493939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c700b36-2b1f-4045-a8c3-73726a180957 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:40.494248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c700b36-2b1f-4045-a8c3-73726a180957 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:40.495698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c700b36-2b1f-4045-a8c3-73726a180957 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:40.512937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 520/1072] 10.222.0.25 () {56 vars in 1133 bytes} [Mon May 9 18:17:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:40.518544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb748f-f4c7-4fcd-a189-e16d49dc6e12 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:40.612647 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb748f-f4c7-4fcd-a189-e16d49dc6e12 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:40.644120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12d4dc-94a7-41b4-a787-ed176e7fbbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:40.644906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12d4dc-94a7-41b4-a787-ed176e7fbbff None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:40.645186 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12d4dc-94a7-41b4-a787-ed176e7fbbff None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:40.645459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12d4dc-94a7-41b4-a787-ed176e7fbbff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:40.655439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12d4dc-94a7-41b4-a787-ed176e7fbbff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:40.656957 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12d4dc-94a7-41b4-a787-ed176e7fbbff None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-0a12d4dc-94a7-41b4-a787-ed176e7fbbff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:40.673582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:40.674731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:40.675958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:40.675958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:40.675958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:40.675958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:40.679418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 553/1073] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:40.680420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c700b36-2b1f-4045-a8c3-73726a180957 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:40.775566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c700b36-2b1f-4045-a8c3-73726a180957 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:40.808442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b41a6-7a92-4e6d-932e-c89d54f2bf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:40.809105 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b41a6-7a92-4e6d-932e-c89d54f2bf3f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:40.809389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b41a6-7a92-4e6d-932e-c89d54f2bf3f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:40.809672 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b41a6-7a92-4e6d-932e-c89d54f2bf3f None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:40.810383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b41a6-7a92-4e6d-932e-c89d54f2bf3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:40.810689 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b41a6-7a92-4e6d-932e-c89d54f2bf3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:40.812095 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b41a6-7a92-4e6d-932e-c89d54f2bf3f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:40.822855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 521/1074] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:40.828496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12d4dc-94a7-41b4-a787-ed176e7fbbff None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:40.927238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12d4dc-94a7-41b4-a787-ed176e7fbbff None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:40.960119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed5dda-a9ca-4e06-99b0-4a401b2172fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:40.960913 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed5dda-a9ca-4e06-99b0-4a401b2172fe None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:40.961192 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed5dda-a9ca-4e06-99b0-4a401b2172fe None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:40.961462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed5dda-a9ca-4e06-99b0-4a401b2172fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:40.968025 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed5dda-a9ca-4e06-99b0-4a401b2172fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:40.970004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed5dda-a9ca-4e06-99b0-4a401b2172fe None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-97ed5dda-a9ca-4e06-99b0-4a401b2172fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:17:40.987002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:40.993245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:40.994424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:17:40.994424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:17:40.994424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:17:40.994424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:17:40.994424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:17:40.994424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:17:40.994424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:40.994424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:40.994424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 554/1075] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:40.995117 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b41a6-7a92-4e6d-932e-c89d54f2bf3f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:41.086007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce2346-9920-4f67-976a-2a42349796c5 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:41.086291 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce2346-9920-4f67-976a-2a42349796c5 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:41.086570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce2346-9920-4f67-976a-2a42349796c5 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:41.087684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b41a6-7a92-4e6d-932e-c89d54f2bf3f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:41.087960 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 555/1076] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:17:41.092969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38327f7e-4dc3-4a84-a4ca-ee674449bd97 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:41.093827 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38327f7e-4dc3-4a84-a4ca-ee674449bd97 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:41.094102 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38327f7e-4dc3-4a84-a4ca-ee674449bd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:41.109880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-38327f7e-4dc3-4a84-a4ca-ee674449bd97 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:41.111020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-38327f7e-4dc3-4a84-a4ca-ee674449bd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:41.119385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf44a5-a702-43d5-a39c-0ae182e23904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:41.120039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf44a5-a702-43d5-a39c-0ae182e23904 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:41.120316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf44a5-a702-43d5-a39c-0ae182e23904 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:41.120588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf44a5-a702-43d5-a39c-0ae182e23904 None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:41.121358 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf44a5-a702-43d5-a39c-0ae182e23904 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:41.121660 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf44a5-a702-43d5-a39c-0ae182e23904 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:41.123057 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf44a5-a702-43d5-a39c-0ae182e23904 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:41.134962 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 522/1077] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:17:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:41.141345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf44a5-a702-43d5-a39c-0ae182e23904 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:41.175049 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 556/1078] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:41.196846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38327f7e-4dc3-4a84-a4ca-ee674449bd97 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:41.232889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf44a5-a702-43d5-a39c-0ae182e23904 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:41.264859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed833b7-f543-4f0d-aedc-035a80582512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:41.266268 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed833b7-f543-4f0d-aedc-035a80582512 None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:41.266545 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed833b7-f543-4f0d-aedc-035a80582512 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:41.266815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed833b7-f543-4f0d-aedc-035a80582512 None admin] PATH_INFO: `/v3/projects/4849b58351c74caf92a584686c74e397/users/f769ee9f31354e5296645fb1e3bd84b6/roles/104650f78a264c78a8cf5c6c4b389575` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:41.269023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed833b7-f543-4f0d-aedc-035a80582512 None admin] RBAC: Authorizing `identity:create_grant(project_id=4849b58351c74caf92a584686c74e397, user_id=f769ee9f31354e5296645fb1e3bd84b6, role_id=104650f78a264c78a8cf5c6c4b389575)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:41.270429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed833b7-f543-4f0d-aedc-035a80582512 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:41.283637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 523/1079] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:17:41 2022] PUT /identity/v3/projects/4849b58351c74caf92a584686c74e397/users/f769ee9f31354e5296645fb1e3bd84b6/roles/104650f78a264c78a8cf5c6c4b389575 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:17:41.299344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38327f7e-4dc3-4a84-a4ca-ee674449bd97 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:41.324387 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06d708-4c93-4246-9435-87774d01cd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:41.325197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06d708-4c93-4246-9435-87774d01cd1f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:41.325475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06d708-4c93-4246-9435-87774d01cd1f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:41.325745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06d708-4c93-4246-9435-87774d01cd1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:41.337177 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06d708-4c93-4246-9435-87774d01cd1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-4b06d708-4c93-4246-9435-87774d01cd1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:41.345691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:41.346927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:41.347842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:41.347842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:41.347842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:41.347842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:41.347842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:41.347842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:41.347842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:41.347842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:41.347842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:41.347842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:41.347842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:17:41.347842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:41.347842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 557/1080] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:17:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:41.353400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed833b7-f543-4f0d-aedc-035a80582512 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:41.454154 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed833b7-f543-4f0d-aedc-035a80582512 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:41.488182 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a15180-47cb-40ef-bd7b-32d92cdc9c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:41.488884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a15180-47cb-40ef-bd7b-32d92cdc9c2e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:41.489168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a15180-47cb-40ef-bd7b-32d92cdc9c2e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:41.489442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a15180-47cb-40ef-bd7b-32d92cdc9c2e None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:41.490186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a15180-47cb-40ef-bd7b-32d92cdc9c2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:41.490491 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a15180-47cb-40ef-bd7b-32d92cdc9c2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:41.491876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a15180-47cb-40ef-bd7b-32d92cdc9c2e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:41.500206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 524/1081] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:17:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:41.506297 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06d708-4c93-4246-9435-87774d01cd1f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:41.601190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06d708-4c93-4246-9435-87774d01cd1f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:41.632579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a083bb8-458c-4517-adc0-ae8f2fc3c7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:41.633434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a083bb8-458c-4517-adc0-ae8f2fc3c7ed None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:41.633712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a083bb8-458c-4517-adc0-ae8f2fc3c7ed None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:41.633984 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a083bb8-458c-4517-adc0-ae8f2fc3c7ed None admin] PATH_INFO: `/v3/users/99bbf92cdba043a094585191d88d9ebf` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:41.635148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a083bb8-458c-4517-adc0-ae8f2fc3c7ed None admin] RBAC: Authorizing `identity:get_user(user_id=99bbf92cdba043a094585191d88d9ebf)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:41.636581 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a083bb8-458c-4517-adc0-ae8f2fc3c7ed None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:41.638179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 558/1082] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:17:41 2022] GET /identity/v3/users/99bbf92cdba043a094585191d88d9ebf => generated 285 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:41.644417 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a15180-47cb-40ef-bd7b-32d92cdc9c2e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:41.737935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a15180-47cb-40ef-bd7b-32d92cdc9c2e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:41.770509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe612ebc-c958-463f-8034-f247b6f6cf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:41.771403 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe612ebc-c958-463f-8034-f247b6f6cf49 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:41.771686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe612ebc-c958-463f-8034-f247b6f6cf49 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:41.771959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe612ebc-c958-463f-8034-f247b6f6cf49 None admin] PATH_INFO: `/v3/projects/cd6a6211651144beb028e5b0f088028f` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:41.773134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe612ebc-c958-463f-8034-f247b6f6cf49 None admin] RBAC: Authorizing `identity:get_project(project_id=cd6a6211651144beb028e5b0f088028f)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:41.774551 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe612ebc-c958-463f-8034-f247b6f6cf49 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:41.776278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 525/1083] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:17:41 2022] GET /identity/v3/projects/cd6a6211651144beb028e5b0f088028f => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:41.782378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a083bb8-458c-4517-adc0-ae8f2fc3c7ed None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:41.875803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a083bb8-458c-4517-adc0-ae8f2fc3c7ed None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:41.906682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec045196-8d25-4e0d-9a7e-3145d8544a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:41.907595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec045196-8d25-4e0d-9a7e-3145d8544a63 None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:41.907872 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec045196-8d25-4e0d-9a7e-3145d8544a63 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:41.908140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec045196-8d25-4e0d-9a7e-3145d8544a63 None admin] PATH_INFO: `/v3/projects/cd6a6211651144beb028e5b0f088028f/users/99bbf92cdba043a094585191d88d9ebf/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:41.910052 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec045196-8d25-4e0d-9a7e-3145d8544a63 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd6a6211651144beb028e5b0f088028f, user_id=99bbf92cdba043a094585191d88d9ebf, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:41.911490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec045196-8d25-4e0d-9a7e-3145d8544a63 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:41.926549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 559/1084] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:17:41 2022] PUT /identity/v3/projects/cd6a6211651144beb028e5b0f088028f/users/99bbf92cdba043a094585191d88d9ebf/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:17:43.566739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0227a-4e19-4868-b376-c218b51e3fc6 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:43.567319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0227a-4e19-4868-b376-c218b51e3fc6 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:43.567319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0227a-4e19-4868-b376-c218b51e3fc6 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:43.568679 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 526/1085] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:43.574162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67efcd0-0e26-439e-9c0d-a7143f784a06 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:43.574564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67efcd0-0e26-439e-9c0d-a7143f784a06 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:43.574828 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67efcd0-0e26-439e-9c0d-a7143f784a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:43.591462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b67efcd0-0e26-439e-9c0d-a7143f784a06 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:43.592605 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b67efcd0-0e26-439e-9c0d-a7143f784a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:43.654561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 560/1086] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:43.677972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0227a-4e19-4868-b376-c218b51e3fc6 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:43.774012 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0227a-4e19-4868-b376-c218b51e3fc6 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:43.804965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88be8df-37e1-47c4-87c9-cea450ce8f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:43.805775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88be8df-37e1-47c4-87c9-cea450ce8f09 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:43.806054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88be8df-37e1-47c4-87c9-cea450ce8f09 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:43.806326 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88be8df-37e1-47c4-87c9-cea450ce8f09 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:43.816420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88be8df-37e1-47c4-87c9-cea450ce8f09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-d88be8df-37e1-47c4-87c9-cea450ce8f09 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:43.824524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:43.825636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:43.826501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:43.826501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:43.826501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:43.826501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:43.826501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:43.826501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:43.826501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:43.826501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:43.826501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:43.826501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:43.826501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 18:17:43.826501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:43.826501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 527/1087] 10.222.0.25 () {56 vars in 1120 bytes} [Mon May 9 18:17:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:43.832533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67efcd0-0e26-439e-9c0d-a7143f784a06 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:43.924616 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67efcd0-0e26-439e-9c0d-a7143f784a06 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:43.957609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f59e5c-7571-44a5-877d-4edef32da807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:43.958312 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f59e5c-7571-44a5-877d-4edef32da807 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:43.958591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f59e5c-7571-44a5-877d-4edef32da807 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:43.958862 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f59e5c-7571-44a5-877d-4edef32da807 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:43.959668 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f59e5c-7571-44a5-877d-4edef32da807 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:43.959968 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f59e5c-7571-44a5-877d-4edef32da807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:43.961893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f59e5c-7571-44a5-877d-4edef32da807 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:43.970754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 561/1088] 10.222.0.25 () {56 vars in 1115 bytes} [Mon May 9 18:17:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:43.976357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88be8df-37e1-47c4-87c9-cea450ce8f09 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:44.070519 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88be8df-37e1-47c4-87c9-cea450ce8f09 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:44.101919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5837c-2e42-4fa2-89c2-9801dd00cd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:44.102639 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5837c-2e42-4fa2-89c2-9801dd00cd41 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:44.102917 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5837c-2e42-4fa2-89c2-9801dd00cd41 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:44.103199 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5837c-2e42-4fa2-89c2-9801dd00cd41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:44.113657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd5837c-2e42-4fa2-89c2-9801dd00cd41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:44.115069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd5837c-2e42-4fa2-89c2-9801dd00cd41 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-abd5837c-2e42-4fa2-89c2-9801dd00cd41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:44.131903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:44.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:44.133863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:44.133863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:44.133863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:44.133863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:44.133863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 528/1089] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:17:44.138799 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f59e5c-7571-44a5-877d-4edef32da807 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:44.140244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c11a5-b732-432c-8030-3228f2ae64f9 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:44.140528 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c11a5-b732-432c-8030-3228f2ae64f9 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:44.140808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c11a5-b732-432c-8030-3228f2ae64f9 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:44.142110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 529/1090] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:17:44.147092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5c4bb-de1d-4db1-9c94-864d71e0cf23 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:44.147426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5c4bb-de1d-4db1-9c94-864d71e0cf23 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:44.147700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5c4bb-de1d-4db1-9c94-864d71e0cf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:44.164426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-6ec5c4bb-de1d-4db1-9c94-864d71e0cf23 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:44.165578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-6ec5c4bb-de1d-4db1-9c94-864d71e0cf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:44.227903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 530/1091] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:17:44.234331 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f59e5c-7571-44a5-877d-4edef32da807 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:44.251366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5c4bb-de1d-4db1-9c94-864d71e0cf23 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:44.266768 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b724ed-c4f7-46fa-831b-7ee76332bf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:44.267461 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b724ed-c4f7-46fa-831b-7ee76332bf1e None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:44.267748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b724ed-c4f7-46fa-831b-7ee76332bf1e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:44.268007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b724ed-c4f7-46fa-831b-7ee76332bf1e None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:44.268706 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b724ed-c4f7-46fa-831b-7ee76332bf1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:44.269001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b724ed-c4f7-46fa-831b-7ee76332bf1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:44.270333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b724ed-c4f7-46fa-831b-7ee76332bf1e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:44.281319 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 562/1092] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:44.286883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b724ed-c4f7-46fa-831b-7ee76332bf1e None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:44.342881 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5c4bb-de1d-4db1-9c94-864d71e0cf23 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:44.375565 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e61c6-61bb-4f43-bb65-9ececb7b31d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:44.376288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e61c6-61bb-4f43-bb65-9ececb7b31d1 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:44.376822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e61c6-61bb-4f43-bb65-9ececb7b31d1 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:44.377089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e61c6-61bb-4f43-bb65-9ececb7b31d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:44.380220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b724ed-c4f7-46fa-831b-7ee76332bf1e None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:44.386626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e61c6-61bb-4f43-bb65-9ececb7b31d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-069e61c6-61bb-4f43-bb65-9ececb7b31d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:17:44.394444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:44.395494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:44.396527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:44.396527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:44.396527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:44.396527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:44.396527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:17:44.396527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:17:44.396527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:17:44.396527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:17:44.396527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:17:44.396527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:17:44.396527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:17:44.396527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:17:44.396527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 531/1093] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:17:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:44.401506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e61c6-61bb-4f43-bb65-9ececb7b31d1 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:44.412331 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac592341-d1dc-4e6f-9205-02a76c21ed23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:44.413117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac592341-d1dc-4e6f-9205-02a76c21ed23 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:44.413390 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac592341-d1dc-4e6f-9205-02a76c21ed23 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:44.413659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac592341-d1dc-4e6f-9205-02a76c21ed23 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:44.420339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac592341-d1dc-4e6f-9205-02a76c21ed23 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:44.422577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac592341-d1dc-4e6f-9205-02a76c21ed23 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-ac592341-d1dc-4e6f-9205-02a76c21ed23 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:17:44.435932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:44.437529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:44.438477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:44.438477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:44.438477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:17:44.438477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:17:44.438477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:17:44.438477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:17:44.438477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:17:44.438477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:17:44.438477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 18:17:44.438477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:44.438477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 563/1094] 10.222.0.25 () {56 vars in 1093 bytes} [Mon May 9 18:17:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:44.443147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac592341-d1dc-4e6f-9205-02a76c21ed23 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:44.495418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e61c6-61bb-4f43-bb65-9ececb7b31d1 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:44.526375 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbad2d-d039-433c-8b69-9d04e75c433b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:44.527033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbad2d-d039-433c-8b69-9d04e75c433b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:44.527341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbad2d-d039-433c-8b69-9d04e75c433b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:44.527625 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbad2d-d039-433c-8b69-9d04e75c433b None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:44.528358 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbad2d-d039-433c-8b69-9d04e75c433b None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:44.528661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbad2d-d039-433c-8b69-9d04e75c433b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:44.530016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbad2d-d039-433c-8b69-9d04e75c433b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:44.533591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac592341-d1dc-4e6f-9205-02a76c21ed23 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:44.538174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 532/1095] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:17:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:44.548964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbad2d-d039-433c-8b69-9d04e75c433b None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:44.568474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f5338-741b-409f-bd1d-b415a58423fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:44.569143 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f5338-741b-409f-bd1d-b415a58423fd None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:44.569422 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f5338-741b-409f-bd1d-b415a58423fd None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:44.569696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f5338-741b-409f-bd1d-b415a58423fd None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:44.570514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f5338-741b-409f-bd1d-b415a58423fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:44.570815 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f5338-741b-409f-bd1d-b415a58423fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:44.572290 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f5338-741b-409f-bd1d-b415a58423fd None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:44.588497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 564/1096] 10.222.0.25 () {56 vars in 1133 bytes} [Mon May 9 18:17:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:44.594562 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f5338-741b-409f-bd1d-b415a58423fd None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:44.645676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbad2d-d039-433c-8b69-9d04e75c433b None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:44.677392 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be70842-474e-4b47-bf61-22b60441d379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:44.678251 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be70842-474e-4b47-bf61-22b60441d379 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:44.678539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be70842-474e-4b47-bf61-22b60441d379 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:44.678812 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be70842-474e-4b47-bf61-22b60441d379 None admin] PATH_INFO: `/v3/users/99bbf92cdba043a094585191d88d9ebf` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:44.680017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be70842-474e-4b47-bf61-22b60441d379 None admin] RBAC: Authorizing `identity:get_user(user_id=99bbf92cdba043a094585191d88d9ebf)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:44.681431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be70842-474e-4b47-bf61-22b60441d379 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:44.683039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 533/1097] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:17:44 2022] GET /identity/v3/users/99bbf92cdba043a094585191d88d9ebf => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:44.685616 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f5338-741b-409f-bd1d-b415a58423fd None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:44.688759 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be70842-474e-4b47-bf61-22b60441d379 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:44.717205 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e689ef4-6462-4c1a-89d3-ba0b9e8f4e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:44.717844 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e689ef4-6462-4c1a-89d3-ba0b9e8f4e21 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:44.718660 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e689ef4-6462-4c1a-89d3-ba0b9e8f4e21 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:44.718935 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e689ef4-6462-4c1a-89d3-ba0b9e8f4e21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:44.779597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be70842-474e-4b47-bf61-22b60441d379 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:44.781252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e689ef4-6462-4c1a-89d3-ba0b9e8f4e21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:44.782681 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e689ef4-6462-4c1a-89d3-ba0b9e8f4e21 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:44.812368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f37ca4-dd14-4df8-a2e4-f0d102536349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:44.813014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f37ca4-dd14-4df8-a2e4-f0d102536349 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:44.813299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f37ca4-dd14-4df8-a2e4-f0d102536349 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:44.813575 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f37ca4-dd14-4df8-a2e4-f0d102536349 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:44.873986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 565/1098] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:17:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 18:17:44.876769 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f37ca4-dd14-4df8-a2e4-f0d102536349 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:44.878199 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f37ca4-dd14-4df8-a2e4-f0d102536349 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:44.879925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e689ef4-6462-4c1a-89d3-ba0b9e8f4e21 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:44.970845 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e689ef4-6462-4c1a-89d3-ba0b9e8f4e21 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:44.971138 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 534/1099] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:17:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 18:17:44.976894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f37ca4-dd14-4df8-a2e4-f0d102536349 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:45.003973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13634e1a-197c-4909-b750-9e82e858153f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:45.004796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13634e1a-197c-4909-b750-9e82e858153f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:45.005085 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13634e1a-197c-4909-b750-9e82e858153f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:45.005352 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13634e1a-197c-4909-b750-9e82e858153f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:45.015642 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13634e1a-197c-4909-b750-9e82e858153f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:45.017066 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13634e1a-197c-4909-b750-9e82e858153f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-13634e1a-197c-4909-b750-9e82e858153f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:17:45.036674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:45.037724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:45.038609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:17:45.038609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:17:45.038609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:17:45.038609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:45.038609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 566/1100] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:45.044033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13634e1a-197c-4909-b750-9e82e858153f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:45.075231 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f37ca4-dd14-4df8-a2e4-f0d102536349 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:45.107434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227279d-8872-44bd-96d2-f6afd7a62b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:45.108286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227279d-8872-44bd-96d2-f6afd7a62b4d None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:45.108588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227279d-8872-44bd-96d2-f6afd7a62b4d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:45.108862 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227279d-8872-44bd-96d2-f6afd7a62b4d None admin] PATH_INFO: `/v3/projects/cd6a6211651144beb028e5b0f088028f` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:45.110018 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227279d-8872-44bd-96d2-f6afd7a62b4d None admin] RBAC: Authorizing `identity:get_project(project_id=cd6a6211651144beb028e5b0f088028f)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:45.111471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227279d-8872-44bd-96d2-f6afd7a62b4d None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:45.113346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 535/1101] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:17:44 2022] GET /identity/v3/projects/cd6a6211651144beb028e5b0f088028f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:45.119009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227279d-8872-44bd-96d2-f6afd7a62b4d None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:45.137027 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13634e1a-197c-4909-b750-9e82e858153f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:45.170174 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3950841-1ee0-4bfe-a991-aebdf57826c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:45.170928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3950841-1ee0-4bfe-a991-aebdf57826c4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:45.171301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3950841-1ee0-4bfe-a991-aebdf57826c4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:45.171591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3950841-1ee0-4bfe-a991-aebdf57826c4 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:45.172295 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3950841-1ee0-4bfe-a991-aebdf57826c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:45.172607 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3950841-1ee0-4bfe-a991-aebdf57826c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:45.173959 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3950841-1ee0-4bfe-a991-aebdf57826c4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:45.184611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 567/1102] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:45.190106 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3950841-1ee0-4bfe-a991-aebdf57826c4 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:45.211633 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227279d-8872-44bd-96d2-f6afd7a62b4d None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:45.243715 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee543f-c35b-4e76-b0a4-a3931ca301e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:45.244373 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee543f-c35b-4e76-b0a4-a3931ca301e6 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:45.244656 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee543f-c35b-4e76-b0a4-a3931ca301e6 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:45.244934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee543f-c35b-4e76-b0a4-a3931ca301e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:45.245756 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee543f-c35b-4e76-b0a4-a3931ca301e6 None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=cd6a6211651144beb028e5b0f088028f, user.id=99bbf92cdba043a094585191d88d9ebf) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:45.246070 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee543f-c35b-4e76-b0a4-a3931ca301e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:45.247503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee543f-c35b-4e76-b0a4-a3931ca301e6 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:45.253869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 536/1103] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:17:45 2022] GET /identity/v3/role_assignments?user.id=99bbf92cdba043a094585191d88d9ebf&scope.project.id=cd6a6211651144beb028e5b0f088028f&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:45.285401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3950841-1ee0-4bfe-a991-aebdf57826c4 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:45.319975 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6b1ff-9a3c-4834-8748-a076416110e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:45.320784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6b1ff-9a3c-4834-8748-a076416110e3 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:45.321061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6b1ff-9a3c-4834-8748-a076416110e3 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:45.321330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6b1ff-9a3c-4834-8748-a076416110e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:45.329165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6b1ff-9a3c-4834-8748-a076416110e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:45.330617 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6b1ff-9a3c-4834-8748-a076416110e3 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-47e6b1ff-9a3c-4834-8748-a076416110e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:17:45.345650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:17:45.346816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:17:45.347793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:17:45.347793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:17:45.347793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:17:45.347793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:17:45.347793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:17:45.347793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:17:45.347793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:17:45.347793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:17:45.347793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 568/1104] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:17:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:17:45.353188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee543f-c35b-4e76-b0a4-a3931ca301e6 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:45.453162 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee543f-c35b-4e76-b0a4-a3931ca301e6 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:45.484823 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0fafe7-ca4f-4363-b859-2f9e1a90ebae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:45.485419 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fafe7-ca4f-4363-b859-2f9e1a90ebae None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:45.485701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fafe7-ca4f-4363-b859-2f9e1a90ebae None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:45.485976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fafe7-ca4f-4363-b859-2f9e1a90ebae None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:45.486751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0fafe7-ca4f-4363-b859-2f9e1a90ebae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:45.487058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0fafe7-ca4f-4363-b859-2f9e1a90ebae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:45.488498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0fafe7-ca4f-4363-b859-2f9e1a90ebae None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:45.500864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 537/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:17:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:45.506928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6b1ff-9a3c-4834-8748-a076416110e3 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:45.600358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6b1ff-9a3c-4834-8748-a076416110e3 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:45.633403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f900de-8928-4cfa-a1eb-3c84c98fb980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:45.634051 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f900de-8928-4cfa-a1eb-3c84c98fb980 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:45.634327 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f900de-8928-4cfa-a1eb-3c84c98fb980 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:45.634597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f900de-8928-4cfa-a1eb-3c84c98fb980 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:45.635428 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f900de-8928-4cfa-a1eb-3c84c98fb980 None admin] RBAC: Adding query filter params (role.id=104650f78a264c78a8cf5c6c4b389575, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=f769ee9f31354e5296645fb1e3bd84b6) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:45.635734 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f900de-8928-4cfa-a1eb-3c84c98fb980 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:45.637130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f900de-8928-4cfa-a1eb-3c84c98fb980 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:45.642693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 569/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:17:45 2022] GET /identity/v3/role_assignments?user.id=f769ee9f31354e5296645fb1e3bd84b6&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=104650f78a264c78a8cf5c6c4b389575 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:47.506804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82de7e-04f0-43f8-b90c-83c29ffcdecd None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:47.507362 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82de7e-04f0-43f8-b90c-83c29ffcdecd None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:47.507453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82de7e-04f0-43f8-b90c-83c29ffcdecd None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:47.508756 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 538/1107] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:47.514014 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdeac4-a6dd-4d41-8930-ff3fa26b62d7 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:47.514425 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdeac4-a6dd-4d41-8930-ff3fa26b62d7 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:47.514557 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdeac4-a6dd-4d41-8930-ff3fa26b62d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:47.533106 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-e0fdeac4-a6dd-4d41-8930-ff3fa26b62d7 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:47.534266 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e0fdeac4-a6dd-4d41-8930-ff3fa26b62d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:47.597149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 570/1108] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:47.620834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82de7e-04f0-43f8-b90c-83c29ffcdecd None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:47.714312 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82de7e-04f0-43f8-b90c-83c29ffcdecd None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:47.744845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ebf1d-7bab-4a9f-b466-f630cc19de9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:47.745622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ebf1d-7bab-4a9f-b466-f630cc19de9b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:47.745897 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ebf1d-7bab-4a9f-b466-f630cc19de9b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:47.746166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ebf1d-7bab-4a9f-b466-f630cc19de9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:47.747328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ebf1d-7bab-4a9f-b466-f630cc19de9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:47.748733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ebf1d-7bab-4a9f-b466-f630cc19de9b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:47.750831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 539/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:47.756927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdeac4-a6dd-4d41-8930-ff3fa26b62d7 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:47.849670 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdeac4-a6dd-4d41-8930-ff3fa26b62d7 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:47.881656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503641cf-02fd-4ba6-90b7-975e5938b409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:47.882299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503641cf-02fd-4ba6-90b7-975e5938b409 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:47.882578 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503641cf-02fd-4ba6-90b7-975e5938b409 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:47.882848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503641cf-02fd-4ba6-90b7-975e5938b409 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:47.883554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503641cf-02fd-4ba6-90b7-975e5938b409 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:47.884951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503641cf-02fd-4ba6-90b7-975e5938b409 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:47.903183 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 571/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:17:47 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:17:50.158561 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abd9a0-3997-4521-b0d6-45a59180fe57 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:50.159140 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abd9a0-3997-4521-b0d6-45a59180fe57 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:50.159140 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abd9a0-3997-4521-b0d6-45a59180fe57 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:50.160479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 540/1111] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:50.166864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69de247-e00c-4175-948d-774345a3c492 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:50.167411 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69de247-e00c-4175-948d-774345a3c492 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:50.167937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69de247-e00c-4175-948d-774345a3c492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:50.190746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-f69de247-e00c-4175-948d-774345a3c492 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:50.191937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f69de247-e00c-4175-948d-774345a3c492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:50.254214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 572/1112] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:50.277260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abd9a0-3997-4521-b0d6-45a59180fe57 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:50.370349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abd9a0-3997-4521-b0d6-45a59180fe57 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:50.401651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d3d7b-d91e-4308-ad19-bba6d20b7a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:50.402478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d3d7b-d91e-4308-ad19-bba6d20b7a32 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:50.402754 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d3d7b-d91e-4308-ad19-bba6d20b7a32 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:50.403022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d3d7b-d91e-4308-ad19-bba6d20b7a32 None admin] PATH_INFO: `/v3/roles/9d45ca30c3584deabd69f2cd196ae5cc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:50.404468 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d3d7b-d91e-4308-ad19-bba6d20b7a32 None admin] RBAC: Authorizing `identity:get_role(role_id=9d45ca30c3584deabd69f2cd196ae5cc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:50.406415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d3d7b-d91e-4308-ad19-bba6d20b7a32 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:50.407770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 541/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:17:50 2022] GET /identity/v3/roles/9d45ca30c3584deabd69f2cd196ae5cc => generated 223 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:50.413344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69de247-e00c-4175-948d-774345a3c492 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:50.504155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69de247-e00c-4175-948d-774345a3c492 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:50.536370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43fe52-d5dc-4158-a5fb-066f2978f240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:50.537155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43fe52-d5dc-4158-a5fb-066f2978f240 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:50.537434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43fe52-d5dc-4158-a5fb-066f2978f240 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:50.537704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43fe52-d5dc-4158-a5fb-066f2978f240 None admin] PATH_INFO: `/v3/users/8576241a8efe438d9ab485a682784f9f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:50.549985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43fe52-d5dc-4158-a5fb-066f2978f240 None admin] RBAC: Authorizing `identity:get_user(user_id=8576241a8efe438d9ab485a682784f9f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:50.551551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43fe52-d5dc-4158-a5fb-066f2978f240 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:50.553298 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 573/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:17:50 2022] GET /identity/v3/users/8576241a8efe438d9ab485a682784f9f => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:50.560079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d3d7b-d91e-4308-ad19-bba6d20b7a32 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:50.652626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d3d7b-d91e-4308-ad19-bba6d20b7a32 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:50.684058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5230b54-2a54-4444-a438-34c4777f0faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:50.684685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5230b54-2a54-4444-a438-34c4777f0faa None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:50.684963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5230b54-2a54-4444-a438-34c4777f0faa None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:50.685232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5230b54-2a54-4444-a438-34c4777f0faa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:50.746329 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5230b54-2a54-4444-a438-34c4777f0faa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:50.747793 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5230b54-2a54-4444-a438-34c4777f0faa None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:50.838712 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 542/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:17:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:17:50.844680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43fe52-d5dc-4158-a5fb-066f2978f240 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:50.936260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43fe52-d5dc-4158-a5fb-066f2978f240 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:50.967670 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e8b86-b230-4402-b15c-0216ca552ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:50.968516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e8b86-b230-4402-b15c-0216ca552ed8 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:50.968792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e8b86-b230-4402-b15c-0216ca552ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:50.969060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e8b86-b230-4402-b15c-0216ca552ed8 None admin] PATH_INFO: `/v3/projects/cd6a6211651144beb028e5b0f088028f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:50.970212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e8b86-b230-4402-b15c-0216ca552ed8 None admin] RBAC: Authorizing `identity:get_project(project_id=cd6a6211651144beb028e5b0f088028f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:50.971782 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e8b86-b230-4402-b15c-0216ca552ed8 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:50.973454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 574/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:17:50 2022] GET /identity/v3/projects/cd6a6211651144beb028e5b0f088028f => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:50.979420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5230b54-2a54-4444-a438-34c4777f0faa None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:51.077790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5230b54-2a54-4444-a438-34c4777f0faa None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:51.110064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5804b7-1bad-48f4-97b2-04301d7fd279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:51.110655 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5804b7-1bad-48f4-97b2-04301d7fd279 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:51.110937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5804b7-1bad-48f4-97b2-04301d7fd279 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:51.111237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5804b7-1bad-48f4-97b2-04301d7fd279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:51.112055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5804b7-1bad-48f4-97b2-04301d7fd279 None admin] RBAC: Adding query filter params (role.id=9d45ca30c3584deabd69f2cd196ae5cc, scope.project.id=cd6a6211651144beb028e5b0f088028f, user.id=8576241a8efe438d9ab485a682784f9f) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:51.112363 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5804b7-1bad-48f4-97b2-04301d7fd279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:51.113756 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5804b7-1bad-48f4-97b2-04301d7fd279 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:51.118970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 543/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:17:50 2022] GET /identity/v3/role_assignments?user.id=8576241a8efe438d9ab485a682784f9f&scope.project.id=cd6a6211651144beb028e5b0f088028f&role.id=9d45ca30c3584deabd69f2cd196ae5cc => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:53.289513 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa721a-8f47-42b5-9a0c-7e0e086ba2f6 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:53.290023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa721a-8f47-42b5-9a0c-7e0e086ba2f6 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:53.290131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa721a-8f47-42b5-9a0c-7e0e086ba2f6 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:53.291447 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 575/1118] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:53.296515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab4d1b-93cc-4bf1-9d3d-702f5f594e19 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:53.296882 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab4d1b-93cc-4bf1-9d3d-702f5f594e19 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:53.297069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab4d1b-93cc-4bf1-9d3d-702f5f594e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:53.313169 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-20ab4d1b-93cc-4bf1-9d3d-702f5f594e19 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:53.314330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-20ab4d1b-93cc-4bf1-9d3d-702f5f594e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:53.375630 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 544/1119] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:53.397985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa721a-8f47-42b5-9a0c-7e0e086ba2f6 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:53.492591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa721a-8f47-42b5-9a0c-7e0e086ba2f6 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:53.524891 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6f915-50f4-4635-b1eb-d47e5dfacfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:53.525725 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6f915-50f4-4635-b1eb-d47e5dfacfab None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:53.526006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6f915-50f4-4635-b1eb-d47e5dfacfab None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:53.526277 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6f915-50f4-4635-b1eb-d47e5dfacfab None admin] PATH_INFO: `/v3/roles/9d45ca30c3584deabd69f2cd196ae5cc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:53.527752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6f915-50f4-4635-b1eb-d47e5dfacfab None admin] RBAC: Authorizing `identity:get_role(role_id=9d45ca30c3584deabd69f2cd196ae5cc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:53.529109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6f915-50f4-4635-b1eb-d47e5dfacfab None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:53.530471 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 576/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:17:53 2022] GET /identity/v3/roles/9d45ca30c3584deabd69f2cd196ae5cc => generated 223 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:53.536187 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab4d1b-93cc-4bf1-9d3d-702f5f594e19 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:53.628001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab4d1b-93cc-4bf1-9d3d-702f5f594e19 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:53.659366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ff1f8-de57-469b-8117-18bbfc4c84ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:53.660220 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ff1f8-de57-469b-8117-18bbfc4c84ab None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:53.660502 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ff1f8-de57-469b-8117-18bbfc4c84ab None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:53.660776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ff1f8-de57-469b-8117-18bbfc4c84ab None admin] PATH_INFO: `/v3/users/8576241a8efe438d9ab485a682784f9f` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:53.661939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ff1f8-de57-469b-8117-18bbfc4c84ab None admin] RBAC: Authorizing `identity:get_user(user_id=8576241a8efe438d9ab485a682784f9f)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:53.663368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ff1f8-de57-469b-8117-18bbfc4c84ab None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:53.664942 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 545/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:17:53 2022] GET /identity/v3/users/8576241a8efe438d9ab485a682784f9f => generated 286 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:53.670208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6f915-50f4-4635-b1eb-d47e5dfacfab None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:53.763001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6f915-50f4-4635-b1eb-d47e5dfacfab None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:53.794415 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a96051-55c9-4861-a1cf-78017117ec73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:53.795274 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a96051-55c9-4861-a1cf-78017117ec73 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:53.795586 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a96051-55c9-4861-a1cf-78017117ec73 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:53.795864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a96051-55c9-4861-a1cf-78017117ec73 None admin] PATH_INFO: `/v3/projects/cd6a6211651144beb028e5b0f088028f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:53.797014 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a96051-55c9-4861-a1cf-78017117ec73 None admin] RBAC: Authorizing `identity:get_project(project_id=cd6a6211651144beb028e5b0f088028f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:53.798410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a96051-55c9-4861-a1cf-78017117ec73 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:53.800072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 577/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:17:53 2022] GET /identity/v3/projects/cd6a6211651144beb028e5b0f088028f => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:53.805668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ff1f8-de57-469b-8117-18bbfc4c84ab None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:53.897624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ff1f8-de57-469b-8117-18bbfc4c84ab None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:53.928869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec64b6-3d7f-495c-bdc2-5fff7d6c78fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:53.929724 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec64b6-3d7f-495c-bdc2-5fff7d6c78fe None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:53.930003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec64b6-3d7f-495c-bdc2-5fff7d6c78fe None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:53.930277 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec64b6-3d7f-495c-bdc2-5fff7d6c78fe None admin] PATH_INFO: `/v3/projects/cd6a6211651144beb028e5b0f088028f/users/8576241a8efe438d9ab485a682784f9f/roles/9d45ca30c3584deabd69f2cd196ae5cc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:53.932413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec64b6-3d7f-495c-bdc2-5fff7d6c78fe None admin] RBAC: Authorizing `identity:create_grant(project_id=cd6a6211651144beb028e5b0f088028f, user_id=8576241a8efe438d9ab485a682784f9f, role_id=9d45ca30c3584deabd69f2cd196ae5cc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:53.933800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec64b6-3d7f-495c-bdc2-5fff7d6c78fe None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:53.946535 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 546/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:17:53 2022] PUT /identity/v3/projects/cd6a6211651144beb028e5b0f088028f/users/8576241a8efe438d9ab485a682784f9f/roles/9d45ca30c3584deabd69f2cd196ae5cc => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:17:56.125558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01d284-301c-46ba-b457-d7c897a94fd2 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:56.126076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01d284-301c-46ba-b457-d7c897a94fd2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:56.126076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01d284-301c-46ba-b457-d7c897a94fd2 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:56.127732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 578/1124] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:56.132850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d5691-e5ad-4177-baf4-5da2fe1cfdf3 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:56.133277 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d5691-e5ad-4177-baf4-5da2fe1cfdf3 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:56.133414 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d5691-e5ad-4177-baf4-5da2fe1cfdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:56.150180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-434d5691-e5ad-4177-baf4-5da2fe1cfdf3 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:56.151387 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-434d5691-e5ad-4177-baf4-5da2fe1cfdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:56.214409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 547/1125] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:56.237906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01d284-301c-46ba-b457-d7c897a94fd2 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:56.333391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01d284-301c-46ba-b457-d7c897a94fd2 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:56.365133 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54608a55-a8fb-420e-992f-04057fda8fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:56.365959 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54608a55-a8fb-420e-992f-04057fda8fb8 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:56.366234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54608a55-a8fb-420e-992f-04057fda8fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:56.366501 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54608a55-a8fb-420e-992f-04057fda8fb8 None admin] PATH_INFO: `/v3/roles/9d45ca30c3584deabd69f2cd196ae5cc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:56.367949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54608a55-a8fb-420e-992f-04057fda8fb8 None admin] RBAC: Authorizing `identity:get_role(role_id=9d45ca30c3584deabd69f2cd196ae5cc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:56.369276 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54608a55-a8fb-420e-992f-04057fda8fb8 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:56.370584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 579/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:17:56 2022] GET /identity/v3/roles/9d45ca30c3584deabd69f2cd196ae5cc => generated 223 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:56.376929 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d5691-e5ad-4177-baf4-5da2fe1cfdf3 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:56.468327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d5691-e5ad-4177-baf4-5da2fe1cfdf3 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:56.499759 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036119fe-9c71-4eb2-a3b1-8051ab1a5717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:56.500599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036119fe-9c71-4eb2-a3b1-8051ab1a5717 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:56.500878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036119fe-9c71-4eb2-a3b1-8051ab1a5717 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:56.501149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036119fe-9c71-4eb2-a3b1-8051ab1a5717 None admin] PATH_INFO: `/v3/users/8576241a8efe438d9ab485a682784f9f` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:56.502560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036119fe-9c71-4eb2-a3b1-8051ab1a5717 None admin] RBAC: Authorizing `identity:get_user(user_id=8576241a8efe438d9ab485a682784f9f)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:56.503994 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036119fe-9c71-4eb2-a3b1-8051ab1a5717 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:56.505572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 548/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:17:56 2022] GET /identity/v3/users/8576241a8efe438d9ab485a682784f9f => generated 286 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:17:56.510910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54608a55-a8fb-420e-992f-04057fda8fb8 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:56.601303 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54608a55-a8fb-420e-992f-04057fda8fb8 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:56.632822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425aab20-0ae4-4d99-89f0-c2b61e949bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:56.633452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aab20-0ae4-4d99-89f0-c2b61e949bec None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:56.633730 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aab20-0ae4-4d99-89f0-c2b61e949bec None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:56.634002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aab20-0ae4-4d99-89f0-c2b61e949bec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:56.695148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425aab20-0ae4-4d99-89f0-c2b61e949bec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:56.696588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425aab20-0ae4-4d99-89f0-c2b61e949bec None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:56.786105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 580/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:17:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:17:56.791747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036119fe-9c71-4eb2-a3b1-8051ab1a5717 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:56.883429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036119fe-9c71-4eb2-a3b1-8051ab1a5717 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:56.915165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293b01f-817d-49c7-b197-8de372f2a3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:56.916026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293b01f-817d-49c7-b197-8de372f2a3b2 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:56.916301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293b01f-817d-49c7-b197-8de372f2a3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:56.916565 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293b01f-817d-49c7-b197-8de372f2a3b2 None admin] PATH_INFO: `/v3/projects/cd6a6211651144beb028e5b0f088028f` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:56.917702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293b01f-817d-49c7-b197-8de372f2a3b2 None admin] RBAC: Authorizing `identity:get_project(project_id=cd6a6211651144beb028e5b0f088028f)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:56.919092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293b01f-817d-49c7-b197-8de372f2a3b2 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:56.920776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 549/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:17:56 2022] GET /identity/v3/projects/cd6a6211651144beb028e5b0f088028f => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:56.926409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425aab20-0ae4-4d99-89f0-c2b61e949bec None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:57.017435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425aab20-0ae4-4d99-89f0-c2b61e949bec None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:57.050635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e7370-b82f-4105-89ae-15d2550fa48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:57.051451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e7370-b82f-4105-89ae-15d2550fa48e None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:57.051733 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e7370-b82f-4105-89ae-15d2550fa48e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:57.052007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e7370-b82f-4105-89ae-15d2550fa48e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:57.052820 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e7370-b82f-4105-89ae-15d2550fa48e None admin] RBAC: Adding query filter params (role.id=9d45ca30c3584deabd69f2cd196ae5cc, scope.project.id=cd6a6211651144beb028e5b0f088028f, user.id=8576241a8efe438d9ab485a682784f9f) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:17:57.053128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e7370-b82f-4105-89ae-15d2550fa48e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:57.054531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e7370-b82f-4105-89ae-15d2550fa48e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:57.060229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 581/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:17:56 2022] GET /identity/v3/role_assignments?user.id=8576241a8efe438d9ab485a682784f9f&scope.project.id=cd6a6211651144beb028e5b0f088028f&role.id=9d45ca30c3584deabd69f2cd196ae5cc => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:59.269757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c068ed9-823d-40cb-8f46-3c69ab2721c7 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:59.270302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c068ed9-823d-40cb-8f46-3c69ab2721c7 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:59.270302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c068ed9-823d-40cb-8f46-3c69ab2721c7 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:59.271711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 550/1131] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:17:59.276757 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62eec1e-db0f-49ad-a903-2ac2ba8de1e9 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:59.277184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62eec1e-db0f-49ad-a903-2ac2ba8de1e9 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:59.277291 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62eec1e-db0f-49ad-a903-2ac2ba8de1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:59.293898 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b62eec1e-db0f-49ad-a903-2ac2ba8de1e9 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:17:59.295083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b62eec1e-db0f-49ad-a903-2ac2ba8de1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:17:59.356469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 582/1132] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:17:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:17:59.379400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c068ed9-823d-40cb-8f46-3c69ab2721c7 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:59.474252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c068ed9-823d-40cb-8f46-3c69ab2721c7 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:59.505897 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f229d-adce-4b6b-b628-3847bb317a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:59.506673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f229d-adce-4b6b-b628-3847bb317a05 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:59.506957 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f229d-adce-4b6b-b628-3847bb317a05 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:59.507260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f229d-adce-4b6b-b628-3847bb317a05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:59.508415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f229d-adce-4b6b-b628-3847bb317a05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:59.509839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f229d-adce-4b6b-b628-3847bb317a05 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:59.511447 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 551/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:17:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:17:59.517791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62eec1e-db0f-49ad-a903-2ac2ba8de1e9 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:17:59.607515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62eec1e-db0f-49ad-a903-2ac2ba8de1e9 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:17:59.639017 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e5b3d-881c-4c22-b46f-76d896a79287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:17:59.640234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e5b3d-881c-4c22-b46f-76d896a79287 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:17:59.640516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e5b3d-881c-4c22-b46f-76d896a79287 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:17:59.640791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e5b3d-881c-4c22-b46f-76d896a79287 None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:17:59.641483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e5b3d-881c-4c22-b46f-76d896a79287 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:17:59.642899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e5b3d-881c-4c22-b46f-76d896a79287 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:17:59.665945 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 583/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Mon May 9 18:17:59 2022] POST /identity/v3/projects => generated 311 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:18:01.885664 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f79729-a803-4802-b8bb-de1409620645 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:01.886229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f79729-a803-4802-b8bb-de1409620645 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:01.886229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f79729-a803-4802-b8bb-de1409620645 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:01.887613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 552/1135] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:18:01.893567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e24ee-ae14-4e8b-b42a-d5ea6244151f None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:01.893981 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e24ee-ae14-4e8b-b42a-d5ea6244151f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:01.894119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e24ee-ae14-4e8b-b42a-d5ea6244151f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:01.911010 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-fc9e24ee-ae14-4e8b-b42a-d5ea6244151f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:18:01.912203 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-fc9e24ee-ae14-4e8b-b42a-d5ea6244151f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:18:01.974149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 584/1136] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:18:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:18:01.998119 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f79729-a803-4802-b8bb-de1409620645 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:02.092925 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f79729-a803-4802-b8bb-de1409620645 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:02.124894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d59162-0f09-451f-abeb-0e3ab875a53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:02.125669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59162-0f09-451f-abeb-0e3ab875a53b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:02.125949 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59162-0f09-451f-abeb-0e3ab875a53b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:02.126478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d59162-0f09-451f-abeb-0e3ab875a53b None admin] PATH_INFO: `/v3/domains/default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:02.127649 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d59162-0f09-451f-abeb-0e3ab875a53b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:02.129053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d59162-0f09-451f-abeb-0e3ab875a53b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:02.130619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 553/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:18:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:02.136204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e24ee-ae14-4e8b-b42a-d5ea6244151f None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:02.227402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e24ee-ae14-4e8b-b42a-d5ea6244151f None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:02.259480 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866684f-e579-4f82-868f-a78bc7a6eeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:02.260128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866684f-e579-4f82-868f-a78bc7a6eeef None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:02.260405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866684f-e579-4f82-868f-a78bc7a6eeef None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:02.260677 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866684f-e579-4f82-868f-a78bc7a6eeef None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:02.261358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866684f-e579-4f82-868f-a78bc7a6eeef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:02.262753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866684f-e579-4f82-868f-a78bc7a6eeef None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:02.281483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 585/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:18:02 2022] POST /identity/v3/users => generated 286 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:18:04.504939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c73ef-5233-48b7-9a6c-a5810c2c656e None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:04.505534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c73ef-5233-48b7-9a6c-a5810c2c656e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:04.505534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c73ef-5233-48b7-9a6c-a5810c2c656e None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:04.506834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 554/1139] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:18:04.512114 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee62029-408c-43f7-bf35-80aa042d7190 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:04.512556 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee62029-408c-43f7-bf35-80aa042d7190 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:04.512662 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee62029-408c-43f7-bf35-80aa042d7190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:04.530797 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-7ee62029-408c-43f7-bf35-80aa042d7190 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:18:04.532002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7ee62029-408c-43f7-bf35-80aa042d7190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:18:04.596973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 586/1140] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:18:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:18:04.619986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c73ef-5233-48b7-9a6c-a5810c2c656e None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:04.714651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c73ef-5233-48b7-9a6c-a5810c2c656e None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:04.745971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c3072-d5b4-44a9-b4f6-ad50375e5695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:04.746752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c3072-d5b4-44a9-b4f6-ad50375e5695 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:04.747034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c3072-d5b4-44a9-b4f6-ad50375e5695 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:04.747341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c3072-d5b4-44a9-b4f6-ad50375e5695 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:04.757624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c3072-d5b4-44a9-b4f6-ad50375e5695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-685c3072-d5b4-44a9-b4f6-ad50375e5695 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:18:04.765733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:18:04.766876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:18:04.767995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:18:04.767995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:18:04.767995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:18:04.767995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:18:04.767995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:18:04.767995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:18:04.767995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:18:04.767995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:18:04.767995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:18:04.767995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:18:04.767995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:18:04.767995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:18:04.767995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 555/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:18:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:18:04.772984 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee62029-408c-43f7-bf35-80aa042d7190 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:04.864280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee62029-408c-43f7-bf35-80aa042d7190 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:04.897951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9f67a-2db1-4be4-8e04-fd3d34e9dde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:04.898612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9f67a-2db1-4be4-8e04-fd3d34e9dde0 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:04.898894 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9f67a-2db1-4be4-8e04-fd3d34e9dde0 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:04.899166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9f67a-2db1-4be4-8e04-fd3d34e9dde0 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:04.899929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9f67a-2db1-4be4-8e04-fd3d34e9dde0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:18:04.900233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9f67a-2db1-4be4-8e04-fd3d34e9dde0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:04.902177 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9f67a-2db1-4be4-8e04-fd3d34e9dde0 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:04.911309 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 587/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:18:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:04.917045 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c3072-d5b4-44a9-b4f6-ad50375e5695 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:05.011404 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c3072-d5b4-44a9-b4f6-ad50375e5695 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:05.042190 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca43d4-5372-461d-8221-cf0853f5bfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:05.043045 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca43d4-5372-461d-8221-cf0853f5bfb3 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:05.043417 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca43d4-5372-461d-8221-cf0853f5bfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:05.043688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca43d4-5372-461d-8221-cf0853f5bfb3 None admin] PATH_INFO: `/v3/users/7c4592567134488a8bf5b7bb16e4f950` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:05.055783 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca43d4-5372-461d-8221-cf0853f5bfb3 None admin] RBAC: Authorizing `identity:get_user(user_id=7c4592567134488a8bf5b7bb16e4f950)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:05.057212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca43d4-5372-461d-8221-cf0853f5bfb3 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:05.058808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 556/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:18:04 2022] GET /identity/v3/users/7c4592567134488a8bf5b7bb16e4f950 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:05.065065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9f67a-2db1-4be4-8e04-fd3d34e9dde0 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:05.162575 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9f67a-2db1-4be4-8e04-fd3d34e9dde0 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:05.194084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c942d-13dd-418c-b569-8c0662ba3d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:05.194710 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c942d-13dd-418c-b569-8c0662ba3d8e None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:05.194987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c942d-13dd-418c-b569-8c0662ba3d8e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:05.195282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c942d-13dd-418c-b569-8c0662ba3d8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:05.258227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c942d-13dd-418c-b569-8c0662ba3d8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:05.259672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c942d-13dd-418c-b569-8c0662ba3d8e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:05.349321 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 588/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:18:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:18:05.355116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca43d4-5372-461d-8221-cf0853f5bfb3 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:05.447638 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca43d4-5372-461d-8221-cf0853f5bfb3 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:05.483874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b044d30-e643-4ef5-9ec7-f14820493d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:05.484717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b044d30-e643-4ef5-9ec7-f14820493d61 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:05.485000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b044d30-e643-4ef5-9ec7-f14820493d61 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:05.485275 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b044d30-e643-4ef5-9ec7-f14820493d61 None admin] PATH_INFO: `/v3/projects/5f5ef1763b8a4d5f911eb2452467326e` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:05.486427 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b044d30-e643-4ef5-9ec7-f14820493d61 None admin] RBAC: Authorizing `identity:get_project(project_id=5f5ef1763b8a4d5f911eb2452467326e)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:05.487873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b044d30-e643-4ef5-9ec7-f14820493d61 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:05.489551 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 557/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:18:05 2022] GET /identity/v3/projects/5f5ef1763b8a4d5f911eb2452467326e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:05.495532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c942d-13dd-418c-b569-8c0662ba3d8e None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:05.587436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c942d-13dd-418c-b569-8c0662ba3d8e None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:05.619510 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61289673-3249-48b1-a7ce-3949c472ac33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:05.620074 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61289673-3249-48b1-a7ce-3949c472ac33 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:05.620348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61289673-3249-48b1-a7ce-3949c472ac33 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:05.620614 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61289673-3249-48b1-a7ce-3949c472ac33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:05.621415 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61289673-3249-48b1-a7ce-3949c472ac33 None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=5f5ef1763b8a4d5f911eb2452467326e, user.id=7c4592567134488a8bf5b7bb16e4f950) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:18:05.621716 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61289673-3249-48b1-a7ce-3949c472ac33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:05.623118 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61289673-3249-48b1-a7ce-3949c472ac33 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:05.628509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 589/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:18:05 2022] GET /identity/v3/role_assignments?user.id=7c4592567134488a8bf5b7bb16e4f950&scope.project.id=5f5ef1763b8a4d5f911eb2452467326e&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 259 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:07.786426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ef400-6b3f-47c0-a86f-44e0c5a362e6 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:07.787000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ef400-6b3f-47c0-a86f-44e0c5a362e6 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:07.787000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ef400-6b3f-47c0-a86f-44e0c5a362e6 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:07.788363 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 558/1147] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:18:07.793580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35aaf7f-7a4a-4be5-b869-2fb1ed22220f None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:07.793995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35aaf7f-7a4a-4be5-b869-2fb1ed22220f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:07.794123 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35aaf7f-7a4a-4be5-b869-2fb1ed22220f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:07.810728 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c35aaf7f-7a4a-4be5-b869-2fb1ed22220f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:18:07.811914 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c35aaf7f-7a4a-4be5-b869-2fb1ed22220f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:18:07.874259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 590/1148] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:18:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:18:07.897709 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ef400-6b3f-47c0-a86f-44e0c5a362e6 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:07.993440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ef400-6b3f-47c0-a86f-44e0c5a362e6 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:08.024874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5957b-6ac7-4b1b-948c-5fec586838e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:08.025649 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5957b-6ac7-4b1b-948c-5fec586838e5 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:08.025930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5957b-6ac7-4b1b-948c-5fec586838e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:08.026201 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5957b-6ac7-4b1b-948c-5fec586838e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:08.036358 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5957b-6ac7-4b1b-948c-5fec586838e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-53e5957b-6ac7-4b1b-948c-5fec586838e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:18:08.043938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:18:08.045652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:18:08.046770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:18:08.046770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:18:08.046770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:18:08.046770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:18:08.046770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:18:08.046770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:18:08.046770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:18:08.046770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:18:08.046770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:18:08.046770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:18:08.046770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:18:08.046770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:18:08.046770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 559/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:18:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:18:08.051566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35aaf7f-7a4a-4be5-b869-2fb1ed22220f None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:08.141430 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35aaf7f-7a4a-4be5-b869-2fb1ed22220f None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:08.173272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81fee6-7cdc-489d-afe0-6fee1eec1102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:08.173923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81fee6-7cdc-489d-afe0-6fee1eec1102 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:08.174198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81fee6-7cdc-489d-afe0-6fee1eec1102 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:08.174469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81fee6-7cdc-489d-afe0-6fee1eec1102 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:08.175418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81fee6-7cdc-489d-afe0-6fee1eec1102 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:18:08.175720 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81fee6-7cdc-489d-afe0-6fee1eec1102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:08.177060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81fee6-7cdc-489d-afe0-6fee1eec1102 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:08.185459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 591/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:18:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:08.191535 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5957b-6ac7-4b1b-948c-5fec586838e5 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:08.286807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5957b-6ac7-4b1b-948c-5fec586838e5 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:08.317245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d753ed6-0475-491e-b62b-8182ce6b149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:08.318034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d753ed6-0475-491e-b62b-8182ce6b149b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:08.318316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d753ed6-0475-491e-b62b-8182ce6b149b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:08.318591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d753ed6-0475-491e-b62b-8182ce6b149b None admin] PATH_INFO: `/v3/users/7c4592567134488a8bf5b7bb16e4f950` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:08.319763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d753ed6-0475-491e-b62b-8182ce6b149b None admin] RBAC: Authorizing `identity:get_user(user_id=7c4592567134488a8bf5b7bb16e4f950)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:08.321159 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d753ed6-0475-491e-b62b-8182ce6b149b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:08.322736 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 560/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:18:08 2022] GET /identity/v3/users/7c4592567134488a8bf5b7bb16e4f950 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:08.328697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81fee6-7cdc-489d-afe0-6fee1eec1102 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:08.420582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81fee6-7cdc-489d-afe0-6fee1eec1102 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:08.450511 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319d6d9-560d-4865-bf3d-7fa1a9ba3ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:08.452084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319d6d9-560d-4865-bf3d-7fa1a9ba3ec9 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:08.452523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319d6d9-560d-4865-bf3d-7fa1a9ba3ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:08.452947 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319d6d9-560d-4865-bf3d-7fa1a9ba3ec9 None admin] PATH_INFO: `/v3/projects/5f5ef1763b8a4d5f911eb2452467326e` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:08.454106 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319d6d9-560d-4865-bf3d-7fa1a9ba3ec9 None admin] RBAC: Authorizing `identity:get_project(project_id=5f5ef1763b8a4d5f911eb2452467326e)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:08.455514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319d6d9-560d-4865-bf3d-7fa1a9ba3ec9 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:08.457123 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 592/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:18:08 2022] GET /identity/v3/projects/5f5ef1763b8a4d5f911eb2452467326e => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:08.462855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d753ed6-0475-491e-b62b-8182ce6b149b None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:08.558641 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d753ed6-0475-491e-b62b-8182ce6b149b None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:08.590335 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af6491-e6a0-4717-b1ad-40c47ab9e561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:08.591153 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af6491-e6a0-4717-b1ad-40c47ab9e561 None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:08.591469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af6491-e6a0-4717-b1ad-40c47ab9e561 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:08.591779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af6491-e6a0-4717-b1ad-40c47ab9e561 None admin] PATH_INFO: `/v3/projects/5f5ef1763b8a4d5f911eb2452467326e/users/7c4592567134488a8bf5b7bb16e4f950/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:08.593692 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af6491-e6a0-4717-b1ad-40c47ab9e561 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f5ef1763b8a4d5f911eb2452467326e, user_id=7c4592567134488a8bf5b7bb16e4f950, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:08.595096 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af6491-e6a0-4717-b1ad-40c47ab9e561 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:08.608735 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 561/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:18:08 2022] PUT /identity/v3/projects/5f5ef1763b8a4d5f911eb2452467326e/users/7c4592567134488a8bf5b7bb16e4f950/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:18:10.769849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2214243-d1cb-4434-81a7-f6c46823d24c None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:10.770414 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2214243-d1cb-4434-81a7-f6c46823d24c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:10.770414 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2214243-d1cb-4434-81a7-f6c46823d24c None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:10.771774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 593/1154] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:18:10.776951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bf506-3bea-4a7f-adfe-5c054f8fcd13 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:10.777335 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bf506-3bea-4a7f-adfe-5c054f8fcd13 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:10.777510 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bf506-3bea-4a7f-adfe-5c054f8fcd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:10.794090 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-e69bf506-3bea-4a7f-adfe-5c054f8fcd13 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:18:10.795924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e69bf506-3bea-4a7f-adfe-5c054f8fcd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:18:10.858397 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 562/1155] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:18:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:18:10.881277 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2214243-d1cb-4434-81a7-f6c46823d24c None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:10.976374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2214243-d1cb-4434-81a7-f6c46823d24c None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:11.008254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e182e-7525-4306-a75e-472862a8771b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:11.009030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e182e-7525-4306-a75e-472862a8771b None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:11.009311 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e182e-7525-4306-a75e-472862a8771b None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:11.009583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e182e-7525-4306-a75e-472862a8771b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:11.019627 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e182e-7525-4306-a75e-472862a8771b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-bd2e182e-7525-4306-a75e-472862a8771b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:18:11.027078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:18:11.028196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:18:11.029729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:18:11.029729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:18:11.029729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:18:11.029729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:18:11.029729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:18:11.029729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:18:11.029729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:18:11.029729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:18:11.029729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:18:11.029729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:18:11.029729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:18:11.029729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:18:11.029729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 594/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:18:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:18:11.034737 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bf506-3bea-4a7f-adfe-5c054f8fcd13 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:11.125846 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bf506-3bea-4a7f-adfe-5c054f8fcd13 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:11.158193 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d070013-6b79-44a7-be6f-fb5df7931b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:11.158845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d070013-6b79-44a7-be6f-fb5df7931b49 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:11.159127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d070013-6b79-44a7-be6f-fb5df7931b49 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:11.159430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d070013-6b79-44a7-be6f-fb5df7931b49 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:11.160161 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d070013-6b79-44a7-be6f-fb5df7931b49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:18:11.160474 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d070013-6b79-44a7-be6f-fb5df7931b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:11.161816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d070013-6b79-44a7-be6f-fb5df7931b49 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:11.169333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 563/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:18:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:11.176937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e182e-7525-4306-a75e-472862a8771b None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:11.272839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e182e-7525-4306-a75e-472862a8771b None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:11.304275 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5580a-ca73-4ca9-bfb9-56c9dfc683b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:11.305058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5580a-ca73-4ca9-bfb9-56c9dfc683b4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:11.305333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5580a-ca73-4ca9-bfb9-56c9dfc683b4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:11.305601 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5580a-ca73-4ca9-bfb9-56c9dfc683b4 None admin] PATH_INFO: `/v3/users/7c4592567134488a8bf5b7bb16e4f950` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:11.306740 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5580a-ca73-4ca9-bfb9-56c9dfc683b4 None admin] RBAC: Authorizing `identity:get_user(user_id=7c4592567134488a8bf5b7bb16e4f950)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:11.308170 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5580a-ca73-4ca9-bfb9-56c9dfc683b4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:11.309732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 595/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:18:11 2022] GET /identity/v3/users/7c4592567134488a8bf5b7bb16e4f950 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:11.315422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d070013-6b79-44a7-be6f-fb5df7931b49 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:11.408295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d070013-6b79-44a7-be6f-fb5df7931b49 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:11.438698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d035d75-bee8-43a9-81ba-f4dd542d0962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:11.439353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d035d75-bee8-43a9-81ba-f4dd542d0962 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:11.439631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d035d75-bee8-43a9-81ba-f4dd542d0962 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:11.439900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d035d75-bee8-43a9-81ba-f4dd542d0962 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:11.501567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d035d75-bee8-43a9-81ba-f4dd542d0962 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:11.502968 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d035d75-bee8-43a9-81ba-f4dd542d0962 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:11.593538 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 564/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:18:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:18:11.599418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5580a-ca73-4ca9-bfb9-56c9dfc683b4 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:11.691032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5580a-ca73-4ca9-bfb9-56c9dfc683b4 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:11.723515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6647e9-286a-4e70-8572-a8222fe07247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:11.724349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6647e9-286a-4e70-8572-a8222fe07247 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:11.724630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6647e9-286a-4e70-8572-a8222fe07247 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:11.724902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6647e9-286a-4e70-8572-a8222fe07247 None admin] PATH_INFO: `/v3/projects/5f5ef1763b8a4d5f911eb2452467326e` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:11.726055 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6647e9-286a-4e70-8572-a8222fe07247 None admin] RBAC: Authorizing `identity:get_project(project_id=5f5ef1763b8a4d5f911eb2452467326e)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:11.727484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6647e9-286a-4e70-8572-a8222fe07247 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:11.729128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 596/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:18:11 2022] GET /identity/v3/projects/5f5ef1763b8a4d5f911eb2452467326e => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:11.735079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d035d75-bee8-43a9-81ba-f4dd542d0962 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:11.828278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d035d75-bee8-43a9-81ba-f4dd542d0962 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:11.859447 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74cd0ea-5147-4458-a3fc-ed83ec66297b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:11.860022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cd0ea-5147-4458-a3fc-ed83ec66297b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:11.860298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cd0ea-5147-4458-a3fc-ed83ec66297b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:11.860572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cd0ea-5147-4458-a3fc-ed83ec66297b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:11.861380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74cd0ea-5147-4458-a3fc-ed83ec66297b None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=5f5ef1763b8a4d5f911eb2452467326e, user.id=7c4592567134488a8bf5b7bb16e4f950) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:18:11.861686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74cd0ea-5147-4458-a3fc-ed83ec66297b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:11.863075 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74cd0ea-5147-4458-a3fc-ed83ec66297b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:11.868552 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 565/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:18:11 2022] GET /identity/v3/role_assignments?user.id=7c4592567134488a8bf5b7bb16e4f950&scope.project.id=5f5ef1763b8a4d5f911eb2452467326e&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:14.071442 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d7006-6bcd-4d26-9a18-31cf9fa082f2 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:14.071952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d7006-6bcd-4d26-9a18-31cf9fa082f2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:14.072068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d7006-6bcd-4d26-9a18-31cf9fa082f2 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:14.073330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 597/1162] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:18:14.078971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8ebb4-7a9a-4efd-96e7-b9f71cba482a None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:14.079448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8ebb4-7a9a-4efd-96e7-b9f71cba482a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:14.079558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8ebb4-7a9a-4efd-96e7-b9f71cba482a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:14.096232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ceb8ebb4-7a9a-4efd-96e7-b9f71cba482a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:18:14.097374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ceb8ebb4-7a9a-4efd-96e7-b9f71cba482a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:18:14.158949 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 566/1163] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:18:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:18:14.180445 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d7006-6bcd-4d26-9a18-31cf9fa082f2 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:14.274549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d7006-6bcd-4d26-9a18-31cf9fa082f2 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:14.305944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee45d58-d9b7-4ba6-8d93-3ade95f3bcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:14.306576 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee45d58-d9b7-4ba6-8d93-3ade95f3bcc4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:14.306856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee45d58-d9b7-4ba6-8d93-3ade95f3bcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:14.307131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee45d58-d9b7-4ba6-8d93-3ade95f3bcc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:14.368797 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee45d58-d9b7-4ba6-8d93-3ade95f3bcc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:14.370221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee45d58-d9b7-4ba6-8d93-3ade95f3bcc4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:14.461937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 598/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:18:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:18:14.468415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8ebb4-7a9a-4efd-96e7-b9f71cba482a None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:14.559279 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8ebb4-7a9a-4efd-96e7-b9f71cba482a None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:14.591847 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb964f-1816-4d79-8872-1956fca0a6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:14.592633 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb964f-1816-4d79-8872-1956fca0a6d6 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:14.592909 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb964f-1816-4d79-8872-1956fca0a6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:14.593176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb964f-1816-4d79-8872-1956fca0a6d6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:14.599480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb964f-1816-4d79-8872-1956fca0a6d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:14.600915 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb964f-1816-4d79-8872-1956fca0a6d6 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-92bb964f-1816-4d79-8872-1956fca0a6d6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:18:14.614442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:18:14.615670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:18:14.616682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 567/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Mon May 9 18:18:14 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:18:14.621502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee45d58-d9b7-4ba6-8d93-3ade95f3bcc4 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:14.713831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee45d58-d9b7-4ba6-8d93-3ade95f3bcc4 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:14.744725 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a76b1-613d-4956-b2e6-c1f85a2403d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:14.745371 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a76b1-613d-4956-b2e6-c1f85a2403d4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:14.745652 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a76b1-613d-4956-b2e6-c1f85a2403d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:14.745925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a76b1-613d-4956-b2e6-c1f85a2403d4 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:14.746634 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a76b1-613d-4956-b2e6-c1f85a2403d4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:18:14.746937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a76b1-613d-4956-b2e6-c1f85a2403d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:14.748805 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a76b1-613d-4956-b2e6-c1f85a2403d4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:14.755836 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 599/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Mon May 9 18:18:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:16.940757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d5fef-faab-4305-a4e7-799157d38af5 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:16.941336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d5fef-faab-4305-a4e7-799157d38af5 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:16.941462 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d5fef-faab-4305-a4e7-799157d38af5 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:16.942819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 568/1167] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:18:16.948018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3fbd3-b6d7-4ad1-987d-a18e40010898 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:16.948398 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3fbd3-b6d7-4ad1-987d-a18e40010898 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:16.948574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3fbd3-b6d7-4ad1-987d-a18e40010898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:16.964836 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-02e3fbd3-b6d7-4ad1-987d-a18e40010898 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:18:16.965995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-02e3fbd3-b6d7-4ad1-987d-a18e40010898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:18:17.028190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 600/1168] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:18:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:18:17.051499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d5fef-faab-4305-a4e7-799157d38af5 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:17.145979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d5fef-faab-4305-a4e7-799157d38af5 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:17.176952 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3b819-d814-46fa-8912-7617a5e98b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:17.177586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3b819-d814-46fa-8912-7617a5e98b19 None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:17.177860 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3b819-d814-46fa-8912-7617a5e98b19 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:17.178131 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3b819-d814-46fa-8912-7617a5e98b19 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:17.178850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3b819-d814-46fa-8912-7617a5e98b19 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:17.180209 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3b819-d814-46fa-8912-7617a5e98b19 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:17.194217 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 569/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Mon May 9 18:18:17 2022] POST /identity/v3/domains => generated 267 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:18:19.436628 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc3a22-d5b0-4373-8425-52d9021fedf2 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:19.437217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc3a22-d5b0-4373-8425-52d9021fedf2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:19.437217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc3a22-d5b0-4373-8425-52d9021fedf2 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:19.438516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 601/1170] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:18:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:18:19.443702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3b9b8-b46d-45c4-9125-5de8eade30a9 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:19.444107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3b9b8-b46d-45c4-9125-5de8eade30a9 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:19.444265 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3b9b8-b46d-45c4-9125-5de8eade30a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:19.460473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-9ea3b9b8-b46d-45c4-9125-5de8eade30a9 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:18:19.461658 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9ea3b9b8-b46d-45c4-9125-5de8eade30a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:18:19.528686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 570/1171] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:18:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:18:19.552000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc3a22-d5b0-4373-8425-52d9021fedf2 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:19.646715 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc3a22-d5b0-4373-8425-52d9021fedf2 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:19.677980 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeab24-c762-4753-adbf-7daaa3818303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:19.678821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeab24-c762-4753-adbf-7daaa3818303 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:19.679100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeab24-c762-4753-adbf-7daaa3818303 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:19.679402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeab24-c762-4753-adbf-7daaa3818303 None admin] PATH_INFO: `/v3/domains/68efa992378643b3becd93febe401ec6` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:19.689760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eeab24-c762-4753-adbf-7daaa3818303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=68efa992378643b3becd93febe401ec6)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:19.691137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eeab24-c762-4753-adbf-7daaa3818303 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:19.692776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 602/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Mon May 9 18:18:19 2022] GET /identity/v3/domains/68efa992378643b3becd93febe401ec6 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:19.698382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3b9b8-b46d-45c4-9125-5de8eade30a9 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:19.790864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3b9b8-b46d-45c4-9125-5de8eade30a9 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:19.822992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460eaa83-3eee-41ee-8839-bbec96001a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:19.823727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460eaa83-3eee-41ee-8839-bbec96001a4d None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:19.824013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460eaa83-3eee-41ee-8839-bbec96001a4d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:19.824293 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460eaa83-3eee-41ee-8839-bbec96001a4d None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:19.824990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460eaa83-3eee-41ee-8839-bbec96001a4d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:19.826443 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460eaa83-3eee-41ee-8839-bbec96001a4d None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:19.850558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 571/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Mon May 9 18:18:19 2022] POST /identity/v3/projects => generated 361 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:18:22.074896 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33445d96-c6b2-4416-bb82-9057b9ae35a9 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:22.075983 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33445d96-c6b2-4416-bb82-9057b9ae35a9 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:22.076099 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33445d96-c6b2-4416-bb82-9057b9ae35a9 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:22.077423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 603/1174] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:18:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:18:22.082934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1250030-52c2-473f-a147-c59718825080 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:22.083388 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1250030-52c2-473f-a147-c59718825080 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:22.083515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1250030-52c2-473f-a147-c59718825080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:22.099733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-c1250030-52c2-473f-a147-c59718825080 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:18:22.100900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c1250030-52c2-473f-a147-c59718825080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:18:22.163466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 572/1175] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:18:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:18:22.188069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33445d96-c6b2-4416-bb82-9057b9ae35a9 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:22.285544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33445d96-c6b2-4416-bb82-9057b9ae35a9 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:22.316305 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77cd81-1209-4f49-ba08-c779a0d8582b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:22.317157 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77cd81-1209-4f49-ba08-c779a0d8582b None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:22.317435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77cd81-1209-4f49-ba08-c779a0d8582b None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:22.317708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77cd81-1209-4f49-ba08-c779a0d8582b None admin] PATH_INFO: `/v3/domains/68efa992378643b3becd93febe401ec6` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:22.318861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77cd81-1209-4f49-ba08-c779a0d8582b None admin] RBAC: Authorizing `identity:get_domain(domain_id=68efa992378643b3becd93febe401ec6)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:22.320302 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77cd81-1209-4f49-ba08-c779a0d8582b None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:22.321894 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 604/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Mon May 9 18:18:22 2022] GET /identity/v3/domains/68efa992378643b3becd93febe401ec6 => generated 267 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:22.328327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1250030-52c2-473f-a147-c59718825080 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:22.419710 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1250030-52c2-473f-a147-c59718825080 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:22.452662 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b74a17-15b1-481c-9b9e-338b634146ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:22.453098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b74a17-15b1-481c-9b9e-338b634146ea None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:22.453374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b74a17-15b1-481c-9b9e-338b634146ea None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:22.453649 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b74a17-15b1-481c-9b9e-338b634146ea None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:22.454347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b74a17-15b1-481c-9b9e-338b634146ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:22.455803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b74a17-15b1-481c-9b9e-338b634146ea None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:22.474079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 573/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:18:22 2022] POST /identity/v3/users => generated 311 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:18:24.710495 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db8342-7021-47c7-bf51-3cbce04e0c5b None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:24.711054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db8342-7021-47c7-bf51-3cbce04e0c5b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:24.711054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db8342-7021-47c7-bf51-3cbce04e0c5b None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:24.712402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 605/1178] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:18:24.718188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799baa3-1bac-45f0-aaa2-4e3290a1ad12 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:24.718635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799baa3-1bac-45f0-aaa2-4e3290a1ad12 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:24.718736 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799baa3-1bac-45f0-aaa2-4e3290a1ad12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:24.734829 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-7799baa3-1bac-45f0-aaa2-4e3290a1ad12 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:18:24.736021 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7799baa3-1bac-45f0-aaa2-4e3290a1ad12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:18:24.797988 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 574/1179] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:18:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:18:24.820545 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db8342-7021-47c7-bf51-3cbce04e0c5b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:24.914552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db8342-7021-47c7-bf51-3cbce04e0c5b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:24.944626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaad26-f601-49b0-8608-82911c96a4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:24.945390 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaad26-f601-49b0-8608-82911c96a4ff None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:24.945664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaad26-f601-49b0-8608-82911c96a4ff None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:24.945933 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaad26-f601-49b0-8608-82911c96a4ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:24.956578 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccaad26-f601-49b0-8608-82911c96a4ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-dccaad26-f601-49b0-8608-82911c96a4ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:18:24.964012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:18:24.965171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:18:24.967104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:18:24.967104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:18:24.967104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:18:24.967104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:18:24.967104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:18:24.967104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:18:24.967104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:18:24.967104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:18:24.967104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:18:24.967104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:18:24.967104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:18:24.967104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:18:24.967104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 606/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:18:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:18:24.971761 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799baa3-1bac-45f0-aaa2-4e3290a1ad12 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:25.063632 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799baa3-1bac-45f0-aaa2-4e3290a1ad12 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:25.095189 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097f4b9-c0bd-481f-9a79-3d8b502164dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:25.095863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097f4b9-c0bd-481f-9a79-3d8b502164dc None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:25.096148 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097f4b9-c0bd-481f-9a79-3d8b502164dc None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:25.096426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097f4b9-c0bd-481f-9a79-3d8b502164dc None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:25.097165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097f4b9-c0bd-481f-9a79-3d8b502164dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:18:25.097470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097f4b9-c0bd-481f-9a79-3d8b502164dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:25.098822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097f4b9-c0bd-481f-9a79-3d8b502164dc None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:25.106371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 575/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:18:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:25.111927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaad26-f601-49b0-8608-82911c96a4ff None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:25.203401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaad26-f601-49b0-8608-82911c96a4ff None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:25.233711 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06583bf-834c-437e-9c6a-a7d3c8180eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:25.234494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06583bf-834c-437e-9c6a-a7d3c8180eed None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:25.234776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06583bf-834c-437e-9c6a-a7d3c8180eed None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:25.235047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06583bf-834c-437e-9c6a-a7d3c8180eed None admin] PATH_INFO: `/v3/users/3d834238b1814bbdb42f8bff47d1199b` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:25.247012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06583bf-834c-437e-9c6a-a7d3c8180eed None admin] RBAC: Authorizing `identity:get_user(user_id=3d834238b1814bbdb42f8bff47d1199b)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:25.248967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06583bf-834c-437e-9c6a-a7d3c8180eed None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:25.250776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 607/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:18:25 2022] GET /identity/v3/users/3d834238b1814bbdb42f8bff47d1199b => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:25.256267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097f4b9-c0bd-481f-9a79-3d8b502164dc None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:25.357315 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097f4b9-c0bd-481f-9a79-3d8b502164dc None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:25.389293 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca343b-1d7a-4c79-8db3-691926e7bc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:25.389934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca343b-1d7a-4c79-8db3-691926e7bc2f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:25.390220 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca343b-1d7a-4c79-8db3-691926e7bc2f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:25.390500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca343b-1d7a-4c79-8db3-691926e7bc2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:25.452994 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca343b-1d7a-4c79-8db3-691926e7bc2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:25.454418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca343b-1d7a-4c79-8db3-691926e7bc2f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:25.545370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 576/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:18:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:18:25.551286 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06583bf-834c-437e-9c6a-a7d3c8180eed None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:25.642915 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06583bf-834c-437e-9c6a-a7d3c8180eed None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:25.675162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a811668-1b2f-4174-a47c-37f2a8592005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:25.676022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a811668-1b2f-4174-a47c-37f2a8592005 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:25.676302 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a811668-1b2f-4174-a47c-37f2a8592005 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:25.676577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a811668-1b2f-4174-a47c-37f2a8592005 None admin] PATH_INFO: `/v3/projects/421b6b56261a4d5ca03ea1193fbf50bf` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:25.677723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a811668-1b2f-4174-a47c-37f2a8592005 None admin] RBAC: Authorizing `identity:get_project(project_id=421b6b56261a4d5ca03ea1193fbf50bf)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:25.679124 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a811668-1b2f-4174-a47c-37f2a8592005 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:25.680779 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 608/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:18:25 2022] GET /identity/v3/projects/421b6b56261a4d5ca03ea1193fbf50bf => generated 361 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:25.687202 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca343b-1d7a-4c79-8db3-691926e7bc2f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:25.779961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca343b-1d7a-4c79-8db3-691926e7bc2f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:25.811994 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552299b-047b-4acf-8976-3e11c348a36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:25.812582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552299b-047b-4acf-8976-3e11c348a36b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:25.812888 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552299b-047b-4acf-8976-3e11c348a36b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:25.813167 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552299b-047b-4acf-8976-3e11c348a36b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:25.813983 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552299b-047b-4acf-8976-3e11c348a36b None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=421b6b56261a4d5ca03ea1193fbf50bf, user.id=3d834238b1814bbdb42f8bff47d1199b) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:18:25.814295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552299b-047b-4acf-8976-3e11c348a36b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:25.815744 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552299b-047b-4acf-8976-3e11c348a36b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:25.820966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 577/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:18:25 2022] GET /identity/v3/role_assignments?user.id=3d834238b1814bbdb42f8bff47d1199b&scope.project.id=421b6b56261a4d5ca03ea1193fbf50bf&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 259 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:27.980668 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36187ceb-ccba-428f-9ebb-bdd09d7884b5 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:27.981134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36187ceb-ccba-428f-9ebb-bdd09d7884b5 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:27.981236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36187ceb-ccba-428f-9ebb-bdd09d7884b5 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:27.982577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 609/1186] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:18:27.987797 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc3ae1-db7a-43b5-adef-14133b4d9d12 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:27.988222 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc3ae1-db7a-43b5-adef-14133b4d9d12 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:27.988354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc3ae1-db7a-43b5-adef-14133b4d9d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:28.006132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-59dc3ae1-db7a-43b5-adef-14133b4d9d12 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:18:28.007378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-59dc3ae1-db7a-43b5-adef-14133b4d9d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:18:28.069825 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 578/1187] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:18:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:18:28.091974 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36187ceb-ccba-428f-9ebb-bdd09d7884b5 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:28.185543 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36187ceb-ccba-428f-9ebb-bdd09d7884b5 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:28.216550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796aaf1-1167-4a0e-8378-e60baedf4017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:28.217267 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796aaf1-1167-4a0e-8378-e60baedf4017 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:28.217543 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796aaf1-1167-4a0e-8378-e60baedf4017 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:28.217814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796aaf1-1167-4a0e-8378-e60baedf4017 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:28.227118 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796aaf1-1167-4a0e-8378-e60baedf4017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-7796aaf1-1167-4a0e-8378-e60baedf4017 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:18:28.234602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:18:28.235800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:18:28.237078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:18:28.237078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:18:28.237078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:18:28.237078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:18:28.237078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:18:28.237078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:18:28.237078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:18:28.237078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:18:28.237078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:18:28.237078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:18:28.237078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:18:28.237078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:18:28.237078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 610/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:18:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:18:28.241516 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc3ae1-db7a-43b5-adef-14133b4d9d12 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:28.334244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc3ae1-db7a-43b5-adef-14133b4d9d12 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:28.365422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a4076-9a98-42ac-9ef7-0b862951d1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:28.366071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a4076-9a98-42ac-9ef7-0b862951d1e1 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:28.366349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a4076-9a98-42ac-9ef7-0b862951d1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:28.366616 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a4076-9a98-42ac-9ef7-0b862951d1e1 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:28.367369 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a4076-9a98-42ac-9ef7-0b862951d1e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:18:28.367668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a4076-9a98-42ac-9ef7-0b862951d1e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:28.369017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a4076-9a98-42ac-9ef7-0b862951d1e1 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:28.377648 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 579/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:18:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:18:28.383425 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796aaf1-1167-4a0e-8378-e60baedf4017 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:28.478832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796aaf1-1167-4a0e-8378-e60baedf4017 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:28.510302 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aadb6a-85f7-447c-8620-71f4c930e0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:28.511148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aadb6a-85f7-447c-8620-71f4c930e0b4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:28.511454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aadb6a-85f7-447c-8620-71f4c930e0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:28.511722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aadb6a-85f7-447c-8620-71f4c930e0b4 None admin] PATH_INFO: `/v3/users/3d834238b1814bbdb42f8bff47d1199b` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:28.512863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aadb6a-85f7-447c-8620-71f4c930e0b4 None admin] RBAC: Authorizing `identity:get_user(user_id=3d834238b1814bbdb42f8bff47d1199b)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:28.514257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aadb6a-85f7-447c-8620-71f4c930e0b4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:28.515888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 611/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:18:28 2022] GET /identity/v3/users/3d834238b1814bbdb42f8bff47d1199b => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:28.521419 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a4076-9a98-42ac-9ef7-0b862951d1e1 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:28.614330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a4076-9a98-42ac-9ef7-0b862951d1e1 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:28.646157 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce9314-7f38-4f4a-a280-7ea06511ee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:28.647005 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce9314-7f38-4f4a-a280-7ea06511ee7c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:28.647336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce9314-7f38-4f4a-a280-7ea06511ee7c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:28.647611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce9314-7f38-4f4a-a280-7ea06511ee7c None admin] PATH_INFO: `/v3/projects/421b6b56261a4d5ca03ea1193fbf50bf` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:28.648767 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce9314-7f38-4f4a-a280-7ea06511ee7c None admin] RBAC: Authorizing `identity:get_project(project_id=421b6b56261a4d5ca03ea1193fbf50bf)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:28.650176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce9314-7f38-4f4a-a280-7ea06511ee7c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:28.651861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 580/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:18:28 2022] GET /identity/v3/projects/421b6b56261a4d5ca03ea1193fbf50bf => generated 361 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:28.657610 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aadb6a-85f7-447c-8620-71f4c930e0b4 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:28.749658 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aadb6a-85f7-447c-8620-71f4c930e0b4 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:28.780587 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3af3db-dc04-4e2f-9c2d-997d65eaf487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:28.781449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3af3db-dc04-4e2f-9c2d-997d65eaf487 None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:28.781729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3af3db-dc04-4e2f-9c2d-997d65eaf487 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:28.782003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3af3db-dc04-4e2f-9c2d-997d65eaf487 None admin] PATH_INFO: `/v3/projects/421b6b56261a4d5ca03ea1193fbf50bf/users/3d834238b1814bbdb42f8bff47d1199b/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:28.783932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3af3db-dc04-4e2f-9c2d-997d65eaf487 None admin] RBAC: Authorizing `identity:create_grant(project_id=421b6b56261a4d5ca03ea1193fbf50bf, user_id=3d834238b1814bbdb42f8bff47d1199b, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:28.785316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3af3db-dc04-4e2f-9c2d-997d65eaf487 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:28.797976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 612/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:18:28 2022] PUT /identity/v3/projects/421b6b56261a4d5ca03ea1193fbf50bf/users/3d834238b1814bbdb42f8bff47d1199b/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:18:30.973956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d2170-07a3-44c8-a153-defb40074076 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:30.974378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d2170-07a3-44c8-a153-defb40074076 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:30.974520 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d2170-07a3-44c8-a153-defb40074076 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:30.975919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 581/1193] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:18:30.981153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7479a8f-0e8e-40be-9d5c-f547fe2c2af6 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:30.982123 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7479a8f-0e8e-40be-9d5c-f547fe2c2af6 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:30.982331 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7479a8f-0e8e-40be-9d5c-f547fe2c2af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:30.998501 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c7479a8f-0e8e-40be-9d5c-f547fe2c2af6 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:18:30.999685 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c7479a8f-0e8e-40be-9d5c-f547fe2c2af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:18:31.062195 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 613/1194] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:18:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:18:31.085233 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d2170-07a3-44c8-a153-defb40074076 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:31.180679 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d2170-07a3-44c8-a153-defb40074076 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:31.211360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07418126-111e-41a8-a27f-3469d069aec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:31.212078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07418126-111e-41a8-a27f-3469d069aec6 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:31.212365 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07418126-111e-41a8-a27f-3469d069aec6 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:31.212642 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07418126-111e-41a8-a27f-3469d069aec6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:31.222552 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07418126-111e-41a8-a27f-3469d069aec6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-07418126-111e-41a8-a27f-3469d069aec6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:18:31.230151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:18:31.231956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:18:31.232982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:18:31.232982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:18:31.232982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:18:31.232982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:18:31.232982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:18:31.232982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:18:31.232982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:18:31.232982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:18:31.232982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:18:31.232982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:18:31.232982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:18:31.232982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:18:31.232982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 582/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:18:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:18:31.237665 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7479a8f-0e8e-40be-9d5c-f547fe2c2af6 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:31.328554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7479a8f-0e8e-40be-9d5c-f547fe2c2af6 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:31.359911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea9dff-cf57-418d-82d0-80d4ec0b1268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:31.360561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea9dff-cf57-418d-82d0-80d4ec0b1268 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:31.360840 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea9dff-cf57-418d-82d0-80d4ec0b1268 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:31.361112 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea9dff-cf57-418d-82d0-80d4ec0b1268 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:31.361848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea9dff-cf57-418d-82d0-80d4ec0b1268 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:18:31.362146 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea9dff-cf57-418d-82d0-80d4ec0b1268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:31.365227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea9dff-cf57-418d-82d0-80d4ec0b1268 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:31.373405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 614/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:18:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:31.379538 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07418126-111e-41a8-a27f-3469d069aec6 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:31.473345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07418126-111e-41a8-a27f-3469d069aec6 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:31.504384 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca6c79-0ada-4245-8fdb-37d54431a25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:31.505250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca6c79-0ada-4245-8fdb-37d54431a25e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:31.505532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca6c79-0ada-4245-8fdb-37d54431a25e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:31.505805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca6c79-0ada-4245-8fdb-37d54431a25e None admin] PATH_INFO: `/v3/users/3d834238b1814bbdb42f8bff47d1199b` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:31.506994 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca6c79-0ada-4245-8fdb-37d54431a25e None admin] RBAC: Authorizing `identity:get_user(user_id=3d834238b1814bbdb42f8bff47d1199b)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:31.508431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca6c79-0ada-4245-8fdb-37d54431a25e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:31.510015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 583/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Mon May 9 18:18:31 2022] GET /identity/v3/users/3d834238b1814bbdb42f8bff47d1199b => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:31.516256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea9dff-cf57-418d-82d0-80d4ec0b1268 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:31.614519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea9dff-cf57-418d-82d0-80d4ec0b1268 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:31.645849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e695af9-e688-44a0-9ac9-282939b5917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:31.646479 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e695af9-e688-44a0-9ac9-282939b5917f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:31.646758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e695af9-e688-44a0-9ac9-282939b5917f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:31.647031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e695af9-e688-44a0-9ac9-282939b5917f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:31.708783 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e695af9-e688-44a0-9ac9-282939b5917f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:31.710204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e695af9-e688-44a0-9ac9-282939b5917f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:31.800203 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 615/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:18:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:18:31.806007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca6c79-0ada-4245-8fdb-37d54431a25e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:31.900509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca6c79-0ada-4245-8fdb-37d54431a25e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:31.930855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929bbaf-36d3-4c5d-aabd-bb03a587767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:31.931727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929bbaf-36d3-4c5d-aabd-bb03a587767e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:31.932009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929bbaf-36d3-4c5d-aabd-bb03a587767e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:31.932292 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929bbaf-36d3-4c5d-aabd-bb03a587767e None admin] PATH_INFO: `/v3/projects/421b6b56261a4d5ca03ea1193fbf50bf` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:31.933444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929bbaf-36d3-4c5d-aabd-bb03a587767e None admin] RBAC: Authorizing `identity:get_project(project_id=421b6b56261a4d5ca03ea1193fbf50bf)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:31.935379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929bbaf-36d3-4c5d-aabd-bb03a587767e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:31.937020 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 584/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Mon May 9 18:18:31 2022] GET /identity/v3/projects/421b6b56261a4d5ca03ea1193fbf50bf => generated 361 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:18:31.942779 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e695af9-e688-44a0-9ac9-282939b5917f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:18:32.034391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e695af9-e688-44a0-9ac9-282939b5917f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:18:32.065897 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bc052-f158-4389-9481-8f13cbe68844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:18:32.066477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bc052-f158-4389-9481-8f13cbe68844 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:18:32.066755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bc052-f158-4389-9481-8f13cbe68844 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:18:32.067026 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bc052-f158-4389-9481-8f13cbe68844 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:18:32.067869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bc052-f158-4389-9481-8f13cbe68844 None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=421b6b56261a4d5ca03ea1193fbf50bf, user.id=3d834238b1814bbdb42f8bff47d1199b) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:18:32.068177 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bc052-f158-4389-9481-8f13cbe68844 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:18:32.069585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bc052-f158-4389-9481-8f13cbe68844 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:18:32.075688 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 616/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:18:31 2022] GET /identity/v3/role_assignments?user.id=3d834238b1814bbdb42f8bff47d1199b&scope.project.id=421b6b56261a4d5ca03ea1193fbf50bf&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 608 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:32.163686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61b8be-2f5b-49d4-a72e-49c44e4fd122 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:32.164256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61b8be-2f5b-49d4-a72e-49c44e4fd122 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:32.164256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61b8be-2f5b-49d4-a72e-49c44e4fd122 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:32.165672 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 585/1201] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:19:32.173599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b91207-81ad-405c-8522-f9b92e593c55 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:32.173599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b91207-81ad-405c-8522-f9b92e593c55 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:32.173599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b91207-81ad-405c-8522-f9b92e593c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:32.189564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-a5b91207-81ad-405c-8522-f9b92e593c55 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:19:32.190754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a5b91207-81ad-405c-8522-f9b92e593c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:19:32.274847 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 617/1202] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:19:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:19:32.300174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61b8be-2f5b-49d4-a72e-49c44e4fd122 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:32.428972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61b8be-2f5b-49d4-a72e-49c44e4fd122 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:32.466385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cd678-e3a6-4894-8680-a578b7c75d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:32.467243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cd678-e3a6-4894-8680-a578b7c75d9d None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:32.467527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cd678-e3a6-4894-8680-a578b7c75d9d None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:32.467840 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cd678-e3a6-4894-8680-a578b7c75d9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:32.484222 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cd678-e3a6-4894-8680-a578b7c75d9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:32.485743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cd678-e3a6-4894-8680-a578b7c75d9d None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-0b3cd678-e3a6-4894-8680-a578b7c75d9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:19:32.504980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:32.512108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:32.514866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:19:32.514866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:19:32.514866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:32.514866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:32.514866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 586/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:32.516086 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b91207-81ad-405c-8522-f9b92e593c55 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:32.670892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b91207-81ad-405c-8522-f9b92e593c55 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:32.708238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f34d1d-06df-413b-b6ff-26e9b1cdf584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:32.708993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f34d1d-06df-413b-b6ff-26e9b1cdf584 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:32.709276 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f34d1d-06df-413b-b6ff-26e9b1cdf584 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:32.709552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f34d1d-06df-413b-b6ff-26e9b1cdf584 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:32.710369 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f34d1d-06df-413b-b6ff-26e9b1cdf584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:32.710680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f34d1d-06df-413b-b6ff-26e9b1cdf584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:32.712284 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f34d1d-06df-413b-b6ff-26e9b1cdf584 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:32.724296 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 618/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:32.730393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cd678-e3a6-4894-8680-a578b7c75d9d None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:32.839695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cd678-e3a6-4894-8680-a578b7c75d9d None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:32.879002 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd38681-fee0-4fad-ac42-df4d8cada755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:32.879719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd38681-fee0-4fad-ac42-df4d8cada755 None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:32.880001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd38681-fee0-4fad-ac42-df4d8cada755 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:32.880276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd38681-fee0-4fad-ac42-df4d8cada755 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:32.880972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd38681-fee0-4fad-ac42-df4d8cada755 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:32.882393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd38681-fee0-4fad-ac42-df4d8cada755 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:32.908484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 587/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Mon May 9 18:19:32 2022] POST /identity/v3/users => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:19:35.531811 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a964f-ceae-4b44-84d9-3d1b44845e0b None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:35.532345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a964f-ceae-4b44-84d9-3d1b44845e0b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:35.532661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a964f-ceae-4b44-84d9-3d1b44845e0b None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:35.534760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 619/1206] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:19:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:19:35.541572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c097a-b9f3-41bd-a1df-09a6e0e8b7d4 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:35.542957 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c097a-b9f3-41bd-a1df-09a6e0e8b7d4 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:35.542957 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c097a-b9f3-41bd-a1df-09a6e0e8b7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:35.560733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-dc5c097a-b9f3-41bd-a1df-09a6e0e8b7d4 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:19:35.562169 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-dc5c097a-b9f3-41bd-a1df-09a6e0e8b7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:19:35.636789 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 588/1207] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:19:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:19:35.660640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a964f-ceae-4b44-84d9-3d1b44845e0b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:35.771715 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a964f-ceae-4b44-84d9-3d1b44845e0b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:35.807586 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f9e45-1628-4e45-9afa-078ef0453b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:35.808322 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f9e45-1628-4e45-9afa-078ef0453b64 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:35.808604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f9e45-1628-4e45-9afa-078ef0453b64 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:35.808897 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f9e45-1628-4e45-9afa-078ef0453b64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:35.820320 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f9e45-1628-4e45-9afa-078ef0453b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-5e7f9e45-1628-4e45-9afa-078ef0453b64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:19:35.828530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:35.829712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:35.830574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:35.830574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:35.830574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:35.830574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:35.830574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:19:35.830574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:19:35.830574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:19:35.830574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:19:35.830574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:19:35.830574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:19:35.830574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:19:35.830574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:35.830574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 620/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:19:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:35.838331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c097a-b9f3-41bd-a1df-09a6e0e8b7d4 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:35.949703 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c097a-b9f3-41bd-a1df-09a6e0e8b7d4 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:35.989475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265e45a-6dd5-44ef-8fac-ae1804b6e6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:35.990200 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265e45a-6dd5-44ef-8fac-ae1804b6e6f9 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:35.990482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265e45a-6dd5-44ef-8fac-ae1804b6e6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:35.990761 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265e45a-6dd5-44ef-8fac-ae1804b6e6f9 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:35.991622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265e45a-6dd5-44ef-8fac-ae1804b6e6f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:35.991976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265e45a-6dd5-44ef-8fac-ae1804b6e6f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:35.993472 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265e45a-6dd5-44ef-8fac-ae1804b6e6f9 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:36.006854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 589/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:19:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:19:36.008232 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f9e45-1628-4e45-9afa-078ef0453b64 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:36.111644 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f9e45-1628-4e45-9afa-078ef0453b64 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:36.147194 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a24405-eb23-4c4e-b948-bc8ae846da42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:36.148042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a24405-eb23-4c4e-b948-bc8ae846da42 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:36.148435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a24405-eb23-4c4e-b948-bc8ae846da42 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:36.148615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a24405-eb23-4c4e-b948-bc8ae846da42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:36.159952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a24405-eb23-4c4e-b948-bc8ae846da42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:36.161415 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a24405-eb23-4c4e-b948-bc8ae846da42 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-71a24405-eb23-4c4e-b948-bc8ae846da42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:19:36.179482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:36.181487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:36.182412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:19:36.182412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:19:36.182412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:36.182412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:36.182412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 621/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:36.187432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265e45a-6dd5-44ef-8fac-ae1804b6e6f9 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:36.300517 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265e45a-6dd5-44ef-8fac-ae1804b6e6f9 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:36.338812 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9124d-749e-44f9-bc53-0b266b0c43d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:36.339836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9124d-749e-44f9-bc53-0b266b0c43d3 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:36.339836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9124d-749e-44f9-bc53-0b266b0c43d3 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:36.340110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9124d-749e-44f9-bc53-0b266b0c43d3 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:36.340795 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9124d-749e-44f9-bc53-0b266b0c43d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:36.341106 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9124d-749e-44f9-bc53-0b266b0c43d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:36.342636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9124d-749e-44f9-bc53-0b266b0c43d3 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:36.355699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 590/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:36.361427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a24405-eb23-4c4e-b948-bc8ae846da42 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:36.468734 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a24405-eb23-4c4e-b948-bc8ae846da42 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:36.506324 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18fcc7b-65b6-4fb9-998d-a34c46e5df88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:36.507144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fcc7b-65b6-4fb9-998d-a34c46e5df88 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:36.507459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fcc7b-65b6-4fb9-998d-a34c46e5df88 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:36.507734 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fcc7b-65b6-4fb9-998d-a34c46e5df88 None admin] PATH_INFO: `/v3/users/placement` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:36.517102 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18fcc7b-65b6-4fb9-998d-a34c46e5df88 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:36.518746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18fcc7b-65b6-4fb9-998d-a34c46e5df88 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-b18fcc7b-65b6-4fb9-998d-a34c46e5df88 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:19:36.534475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:36.536156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:36.537121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:36.537121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:36.537121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:19:36.537121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:19:36.537121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:19:36.537121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:19:36.537121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:19:36.537121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:19:36.537121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 18:19:36.537121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:36.537121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 622/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:36.541884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9124d-749e-44f9-bc53-0b266b0c43d3 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:36.669877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9124d-749e-44f9-bc53-0b266b0c43d3 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:36.710678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26f5d0-792d-4209-8c91-b126142c0573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:36.711415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26f5d0-792d-4209-8c91-b126142c0573 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:36.711706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26f5d0-792d-4209-8c91-b126142c0573 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:36.711980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26f5d0-792d-4209-8c91-b126142c0573 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:36.712933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26f5d0-792d-4209-8c91-b126142c0573 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:36.713255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26f5d0-792d-4209-8c91-b126142c0573 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:36.714802 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26f5d0-792d-4209-8c91-b126142c0573 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:36.734566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 591/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Mon May 9 18:19:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:36.740346 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18fcc7b-65b6-4fb9-998d-a34c46e5df88 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:36.844788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18fcc7b-65b6-4fb9-998d-a34c46e5df88 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:36.882347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec974d-20b4-4a55-836e-eee8936c4bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:36.883326 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec974d-20b4-4a55-836e-eee8936c4bbb None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:36.883684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec974d-20b4-4a55-836e-eee8936c4bbb None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:36.883976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec974d-20b4-4a55-836e-eee8936c4bbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:36.949588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec974d-20b4-4a55-836e-eee8936c4bbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:36.951030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec974d-20b4-4a55-836e-eee8936c4bbb None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:37.053156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 623/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:19:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:19:37.059828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26f5d0-792d-4209-8c91-b126142c0573 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:37.169649 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26f5d0-792d-4209-8c91-b126142c0573 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:37.204657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b9ffd-7c6f-474a-97a4-1a741d268e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:37.205677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b9ffd-7c6f-474a-97a4-1a741d268e06 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:37.206043 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b9ffd-7c6f-474a-97a4-1a741d268e06 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:37.206441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b9ffd-7c6f-474a-97a4-1a741d268e06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:37.218681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b9ffd-7c6f-474a-97a4-1a741d268e06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:37.220742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b9ffd-7c6f-474a-97a4-1a741d268e06 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-fb8b9ffd-7c6f-474a-97a4-1a741d268e06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:19:37.240284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:37.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:37.247322 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:19:37.247322 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:19:37.247322 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:37.247322 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:37.247322 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 592/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:37.247682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec974d-20b4-4a55-836e-eee8936c4bbb None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:37.361158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec974d-20b4-4a55-836e-eee8936c4bbb None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:37.394917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff7831-e386-416b-993e-14af9c2dcc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:37.395606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff7831-e386-416b-993e-14af9c2dcc65 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:37.395887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff7831-e386-416b-993e-14af9c2dcc65 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:37.396158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff7831-e386-416b-993e-14af9c2dcc65 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:37.396968 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff7831-e386-416b-993e-14af9c2dcc65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:37.397248 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff7831-e386-416b-993e-14af9c2dcc65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:37.398680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff7831-e386-416b-993e-14af9c2dcc65 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:37.411591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 624/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:37.418429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b9ffd-7c6f-474a-97a4-1a741d268e06 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:37.527969 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b9ffd-7c6f-474a-97a4-1a741d268e06 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:37.567108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00be7a-dc54-41f0-85f1-ad791b25b024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:37.567969 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00be7a-dc54-41f0-85f1-ad791b25b024 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:37.568213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00be7a-dc54-41f0-85f1-ad791b25b024 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:37.568549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00be7a-dc54-41f0-85f1-ad791b25b024 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:37.578179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00be7a-dc54-41f0-85f1-ad791b25b024 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:37.580350 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00be7a-dc54-41f0-85f1-ad791b25b024 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-5e00be7a-dc54-41f0-85f1-ad791b25b024 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:19:37.602034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:37.603082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:37.604294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:19:37.604294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:19:37.604294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:19:37.604294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:19:37.604294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:19:37.604294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:19:37.604294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:19:37.604294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:37.604294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 593/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:19:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:37.609159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff7831-e386-416b-993e-14af9c2dcc65 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:37.722748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff7831-e386-416b-993e-14af9c2dcc65 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:37.757559 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bc6a0-a609-4ca3-bb6b-69d1bfb47baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:37.758254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bc6a0-a609-4ca3-bb6b-69d1bfb47baf None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:37.758534 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bc6a0-a609-4ca3-bb6b-69d1bfb47baf None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:37.758809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bc6a0-a609-4ca3-bb6b-69d1bfb47baf None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:37.760183 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bc6a0-a609-4ca3-bb6b-69d1bfb47baf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:37.760488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bc6a0-a609-4ca3-bb6b-69d1bfb47baf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:37.761894 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bc6a0-a609-4ca3-bb6b-69d1bfb47baf None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:37.773658 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 625/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:19:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:19:37.779792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00be7a-dc54-41f0-85f1-ad791b25b024 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:37.891666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00be7a-dc54-41f0-85f1-ad791b25b024 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:37.926659 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10bbd7-d271-47f2-8125-471ed3ea174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:37.927429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10bbd7-d271-47f2-8125-471ed3ea174c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:37.927715 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10bbd7-d271-47f2-8125-471ed3ea174c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:37.927998 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10bbd7-d271-47f2-8125-471ed3ea174c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:37.928860 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10bbd7-d271-47f2-8125-471ed3ea174c None admin] RBAC: Adding query filter params (role.id=7b2e521b45dc4a98adad996ca0027de6, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=f747dcb955fa499a972c46587d3db294) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:37.929168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10bbd7-d271-47f2-8125-471ed3ea174c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:37.930674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10bbd7-d271-47f2-8125-471ed3ea174c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:37.937483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 594/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:19:37 2022] GET /identity/v3/role_assignments?user.id=f747dcb955fa499a972c46587d3db294&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=7b2e521b45dc4a98adad996ca0027de6 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:40.277376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7f20a-2755-4e82-a045-c0a1a0d0e2fd None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:40.277932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7f20a-2755-4e82-a045-c0a1a0d0e2fd None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:40.277932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7f20a-2755-4e82-a045-c0a1a0d0e2fd None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:40.279470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 626/1220] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:19:40.284648 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a90bae-f47c-40b6-b608-3a0338ff22f3 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:40.285089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a90bae-f47c-40b6-b608-3a0338ff22f3 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:40.285271 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a90bae-f47c-40b6-b608-3a0338ff22f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:40.303652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-d6a90bae-f47c-40b6-b608-3a0338ff22f3 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:19:40.304949 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-d6a90bae-f47c-40b6-b608-3a0338ff22f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:19:40.378054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 595/1221] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:19:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:19:40.404121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7f20a-2755-4e82-a045-c0a1a0d0e2fd None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:40.507522 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7f20a-2755-4e82-a045-c0a1a0d0e2fd None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:40.541507 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab8222-5e66-4704-8237-deba80891589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:40.542252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab8222-5e66-4704-8237-deba80891589 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:40.542535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab8222-5e66-4704-8237-deba80891589 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:40.542810 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab8222-5e66-4704-8237-deba80891589 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:40.552706 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab8222-5e66-4704-8237-deba80891589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-6aab8222-5e66-4704-8237-deba80891589 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:19:40.560625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:40.561606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:40.563045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:40.563045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:40.563045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:40.563045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:40.563045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:19:40.563045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:19:40.563045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:19:40.563045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:19:40.563045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:19:40.563045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:19:40.563045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:19:40.563045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:40.563045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 627/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:19:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:40.568988 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a90bae-f47c-40b6-b608-3a0338ff22f3 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:40.697374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a90bae-f47c-40b6-b608-3a0338ff22f3 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:40.734723 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a5828-6bc4-4b47-8e7c-3cb1375b2d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:40.735558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a5828-6bc4-4b47-8e7c-3cb1375b2d69 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:40.735841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a5828-6bc4-4b47-8e7c-3cb1375b2d69 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:40.736152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a5828-6bc4-4b47-8e7c-3cb1375b2d69 None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:40.737004 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a5828-6bc4-4b47-8e7c-3cb1375b2d69 None admin] RBAC: Adding query filter params (name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:40.737391 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a5828-6bc4-4b47-8e7c-3cb1375b2d69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:40.738971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a5828-6bc4-4b47-8e7c-3cb1375b2d69 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:40.751207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 596/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:19:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:40.755077 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab8222-5e66-4704-8237-deba80891589 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:40.871490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab8222-5e66-4704-8237-deba80891589 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:40.914162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba072cba-b0ed-41da-aa49-bb4e8d8ea401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:40.915169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba072cba-b0ed-41da-aa49-bb4e8d8ea401 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:40.915633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba072cba-b0ed-41da-aa49-bb4e8d8ea401 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:40.915947 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba072cba-b0ed-41da-aa49-bb4e8d8ea401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:40.927821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba072cba-b0ed-41da-aa49-bb4e8d8ea401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:40.929561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba072cba-b0ed-41da-aa49-bb4e8d8ea401 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-ba072cba-b0ed-41da-aa49-bb4e8d8ea401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:19:40.951382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:40.952527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:40.953443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:19:40.953443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:19:40.953443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:40.953443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:40.953443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 628/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:40.958962 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a5828-6bc4-4b47-8e7c-3cb1375b2d69 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:41.070091 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a5828-6bc4-4b47-8e7c-3cb1375b2d69 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:41.107748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e523b3-bc90-42ee-8b5d-19344b71a7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:41.108518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e523b3-bc90-42ee-8b5d-19344b71a7c6 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:41.108799 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e523b3-bc90-42ee-8b5d-19344b71a7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:41.109072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e523b3-bc90-42ee-8b5d-19344b71a7c6 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:41.109924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e523b3-bc90-42ee-8b5d-19344b71a7c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:41.110266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e523b3-bc90-42ee-8b5d-19344b71a7c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:41.111736 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e523b3-bc90-42ee-8b5d-19344b71a7c6 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:41.125992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 597/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:41.132061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba072cba-b0ed-41da-aa49-bb4e8d8ea401 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:41.234573 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba072cba-b0ed-41da-aa49-bb4e8d8ea401 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:41.271981 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a9e8c-1e52-4777-b636-e6af6e9cec15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:41.272701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a9e8c-1e52-4777-b636-e6af6e9cec15 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:41.273008 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a9e8c-1e52-4777-b636-e6af6e9cec15 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:41.273255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a9e8c-1e52-4777-b636-e6af6e9cec15 None admin] PATH_INFO: `/v3/users/placement` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:41.281584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a9e8c-1e52-4777-b636-e6af6e9cec15 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:41.283098 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a9e8c-1e52-4777-b636-e6af6e9cec15 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-707a9e8c-1e52-4777-b636-e6af6e9cec15 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:19:41.300533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:41.301564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:41.302714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:41.302714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:41.302714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:19:41.302714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:19:41.302714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:19:41.302714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:19:41.302714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:19:41.302714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:19:41.302714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 18:19:41.302714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:41.302714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 629/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:41.308684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e523b3-bc90-42ee-8b5d-19344b71a7c6 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:41.422406 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e523b3-bc90-42ee-8b5d-19344b71a7c6 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:41.462971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54ed2f-516b-47a5-9488-5d4ddda8e15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:41.463672 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54ed2f-516b-47a5-9488-5d4ddda8e15e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:41.463954 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54ed2f-516b-47a5-9488-5d4ddda8e15e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:41.464225 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54ed2f-516b-47a5-9488-5d4ddda8e15e None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:41.465047 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54ed2f-516b-47a5-9488-5d4ddda8e15e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:41.465351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54ed2f-516b-47a5-9488-5d4ddda8e15e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:41.466748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54ed2f-516b-47a5-9488-5d4ddda8e15e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:41.483967 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 598/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Mon May 9 18:19:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:41.489560 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a9e8c-1e52-4777-b636-e6af6e9cec15 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:41.596416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a9e8c-1e52-4777-b636-e6af6e9cec15 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:41.630600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56f020-49da-4cb1-9a41-d7f91e5db7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:41.631427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56f020-49da-4cb1-9a41-d7f91e5db7a5 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:41.631713 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56f020-49da-4cb1-9a41-d7f91e5db7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:41.631985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56f020-49da-4cb1-9a41-d7f91e5db7a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:41.642456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56f020-49da-4cb1-9a41-d7f91e5db7a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:41.644015 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56f020-49da-4cb1-9a41-d7f91e5db7a5 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-dc56f020-49da-4cb1-9a41-d7f91e5db7a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:19:41.661474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:41.662527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:41.663775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:19:41.663775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:19:41.663775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:41.663775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:41.663775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 630/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:19:41.669061 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54ed2f-516b-47a5-9488-5d4ddda8e15e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:41.775059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54ed2f-516b-47a5-9488-5d4ddda8e15e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:41.814140 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047de6a-f406-4cb1-87fc-996cd39fdc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:41.815145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047de6a-f406-4cb1-87fc-996cd39fdc28 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:41.815469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047de6a-f406-4cb1-87fc-996cd39fdc28 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:41.815743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047de6a-f406-4cb1-87fc-996cd39fdc28 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:41.816702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047de6a-f406-4cb1-87fc-996cd39fdc28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:41.817135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047de6a-f406-4cb1-87fc-996cd39fdc28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:41.818476 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047de6a-f406-4cb1-87fc-996cd39fdc28 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:41.830776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 599/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:41.836657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56f020-49da-4cb1-9a41-d7f91e5db7a5 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:41.943725 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56f020-49da-4cb1-9a41-d7f91e5db7a5 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:41.981661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a86ec-db12-4de1-b972-0cf0bf79a742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:41.982678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a86ec-db12-4de1-b972-0cf0bf79a742 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:41.982961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a86ec-db12-4de1-b972-0cf0bf79a742 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:41.983258 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a86ec-db12-4de1-b972-0cf0bf79a742 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:41.992014 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a86ec-db12-4de1-b972-0cf0bf79a742 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:41.993869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a86ec-db12-4de1-b972-0cf0bf79a742 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-d74a86ec-db12-4de1-b972-0cf0bf79a742 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:19:42.011384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:42.012900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:42.013941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:19:42.013941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:19:42.013941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:19:42.013941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:19:42.013941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:19:42.013941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:19:42.013941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:19:42.013941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:42.013941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 631/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:19:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:19:42.018800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047de6a-f406-4cb1-87fc-996cd39fdc28 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:42.130536 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047de6a-f406-4cb1-87fc-996cd39fdc28 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:42.165418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13750f42-17c2-43a2-819d-ed898c4eddce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:42.166079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13750f42-17c2-43a2-819d-ed898c4eddce None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:42.166357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13750f42-17c2-43a2-819d-ed898c4eddce None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:42.166625 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13750f42-17c2-43a2-819d-ed898c4eddce None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:42.167413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13750f42-17c2-43a2-819d-ed898c4eddce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:42.167715 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13750f42-17c2-43a2-819d-ed898c4eddce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:42.169123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13750f42-17c2-43a2-819d-ed898c4eddce None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:42.180400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 600/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:19:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:42.186499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a86ec-db12-4de1-b972-0cf0bf79a742 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:42.295921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a86ec-db12-4de1-b972-0cf0bf79a742 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:42.332257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b6bb1-69c9-492f-9bd9-b0d3c51c3ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:42.333194 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b6bb1-69c9-492f-9bd9-b0d3c51c3ef8 None admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:42.333479 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b6bb1-69c9-492f-9bd9-b0d3c51c3ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:42.333757 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b6bb1-69c9-492f-9bd9-b0d3c51c3ef8 None admin] PATH_INFO: `/v3/projects/4849b58351c74caf92a584686c74e397/users/f747dcb955fa499a972c46587d3db294/roles/7b2e521b45dc4a98adad996ca0027de6` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:42.348455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b6bb1-69c9-492f-9bd9-b0d3c51c3ef8 None admin] RBAC: Authorizing `identity:create_grant(project_id=4849b58351c74caf92a584686c74e397, user_id=f747dcb955fa499a972c46587d3db294, role_id=7b2e521b45dc4a98adad996ca0027de6)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:42.350096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b6bb1-69c9-492f-9bd9-b0d3c51c3ef8 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:42.366934 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 632/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:19:42 2022] PUT /identity/v3/projects/4849b58351c74caf92a584686c74e397/users/f747dcb955fa499a972c46587d3db294/roles/7b2e521b45dc4a98adad996ca0027de6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:19:44.691347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d83b2-47ac-46ba-a31b-b93d7918babf None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:44.691904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d83b2-47ac-46ba-a31b-b93d7918babf None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:44.692149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d83b2-47ac-46ba-a31b-b93d7918babf None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:44.693930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 601/1233] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:19:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:19:44.699702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db12b49-ee82-42a4-a476-543ac873592c None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:44.700131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db12b49-ee82-42a4-a476-543ac873592c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:44.700236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db12b49-ee82-42a4-a476-543ac873592c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:44.717332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2db12b49-ee82-42a4-a476-543ac873592c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:19:44.718509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2db12b49-ee82-42a4-a476-543ac873592c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:19:44.786074 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 633/1234] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:19:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:19:44.809895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d83b2-47ac-46ba-a31b-b93d7918babf None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:44.914711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d83b2-47ac-46ba-a31b-b93d7918babf None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:44.949096 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afede84-f73c-4cc7-afd5-7fcaff93b7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:44.950055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afede84-f73c-4cc7-afd5-7fcaff93b7ef None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:44.950464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afede84-f73c-4cc7-afd5-7fcaff93b7ef None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:44.950789 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afede84-f73c-4cc7-afd5-7fcaff93b7ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:44.967554 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afede84-f73c-4cc7-afd5-7fcaff93b7ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-8afede84-f73c-4cc7-afd5-7fcaff93b7ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:19:44.975483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:44.976719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:44.977987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:44.977987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:44.977987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:44.977987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:44.977987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:19:44.977987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:19:44.977987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:19:44.977987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:19:44.977987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:19:44.977987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:19:44.977987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 18:19:44.977987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:44.977987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 602/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Mon May 9 18:19:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:44.982905 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db12b49-ee82-42a4-a476-543ac873592c None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:45.085955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db12b49-ee82-42a4-a476-543ac873592c None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:45.123961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1f2cd-e989-4074-9a04-0473ecc4aa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:45.124630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1f2cd-e989-4074-9a04-0473ecc4aa4a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:45.124914 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1f2cd-e989-4074-9a04-0473ecc4aa4a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:45.125195 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1f2cd-e989-4074-9a04-0473ecc4aa4a None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:45.125929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1f2cd-e989-4074-9a04-0473ecc4aa4a None admin] RBAC: Adding query filter params (name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:45.126305 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1f2cd-e989-4074-9a04-0473ecc4aa4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:45.127780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1f2cd-e989-4074-9a04-0473ecc4aa4a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:45.137169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 634/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Mon May 9 18:19:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:45.143259 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afede84-f73c-4cc7-afd5-7fcaff93b7ef None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:45.244919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afede84-f73c-4cc7-afd5-7fcaff93b7ef None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:45.279025 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b24537-bdc0-4cb8-bf95-a466b6f6da19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:45.279932 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24537-bdc0-4cb8-bf95-a466b6f6da19 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:45.280219 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24537-bdc0-4cb8-bf95-a466b6f6da19 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:45.280492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24537-bdc0-4cb8-bf95-a466b6f6da19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:45.290763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b24537-bdc0-4cb8-bf95-a466b6f6da19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:45.292228 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b24537-bdc0-4cb8-bf95-a466b6f6da19 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-e7b24537-bdc0-4cb8-bf95-a466b6f6da19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:19:45.310001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:45.311055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:45.312029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:19:45.312029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:19:45.312029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:45.312029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:45.312029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 603/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:45.317286 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1f2cd-e989-4074-9a04-0473ecc4aa4a None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:45.418143 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1f2cd-e989-4074-9a04-0473ecc4aa4a None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:45.452681 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749823cb-16a9-4a19-a686-aacf7c006de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:45.453307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749823cb-16a9-4a19-a686-aacf7c006de2 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:45.453569 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749823cb-16a9-4a19-a686-aacf7c006de2 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:45.453854 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749823cb-16a9-4a19-a686-aacf7c006de2 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:45.454566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749823cb-16a9-4a19-a686-aacf7c006de2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:45.454866 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749823cb-16a9-4a19-a686-aacf7c006de2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:45.456288 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749823cb-16a9-4a19-a686-aacf7c006de2 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:45.467965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 635/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:45.473805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b24537-bdc0-4cb8-bf95-a466b6f6da19 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:45.579874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b24537-bdc0-4cb8-bf95-a466b6f6da19 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:45.618245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb284c8-bba0-4d2a-b61a-b17581e3dfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:45.619063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb284c8-bba0-4d2a-b61a-b17581e3dfed None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:45.619532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb284c8-bba0-4d2a-b61a-b17581e3dfed None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:45.619819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb284c8-bba0-4d2a-b61a-b17581e3dfed None admin] PATH_INFO: `/v3/users/placement` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:45.629890 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb284c8-bba0-4d2a-b61a-b17581e3dfed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:45.631557 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb284c8-bba0-4d2a-b61a-b17581e3dfed None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-8fb284c8-bba0-4d2a-b61a-b17581e3dfed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:19:45.646893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:45.648130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:45.649288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:45.649288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:45.649288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:19:45.649288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:19:45.649288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:19:45.649288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:19:45.649288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:19:45.649288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:19:45.649288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 18:19:45.649288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:45.649288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 604/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:45.654627 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749823cb-16a9-4a19-a686-aacf7c006de2 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:45.757233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749823cb-16a9-4a19-a686-aacf7c006de2 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:45.792316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a181ea-851f-4ec4-944c-37fadef18de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:45.792979 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a181ea-851f-4ec4-944c-37fadef18de6 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:45.793265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a181ea-851f-4ec4-944c-37fadef18de6 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:45.793530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a181ea-851f-4ec4-944c-37fadef18de6 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:45.794344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a181ea-851f-4ec4-944c-37fadef18de6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:45.794647 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a181ea-851f-4ec4-944c-37fadef18de6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:45.796090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a181ea-851f-4ec4-944c-37fadef18de6 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:45.813071 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 636/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Mon May 9 18:19:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:45.818982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb284c8-bba0-4d2a-b61a-b17581e3dfed None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:45.920616 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb284c8-bba0-4d2a-b61a-b17581e3dfed None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:45.954623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc4d6a-99d4-485e-b016-b0bd49c1a899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:45.955308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc4d6a-99d4-485e-b016-b0bd49c1a899 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:45.955611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc4d6a-99d4-485e-b016-b0bd49c1a899 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:45.955889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc4d6a-99d4-485e-b016-b0bd49c1a899 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:46.028176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc4d6a-99d4-485e-b016-b0bd49c1a899 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:46.029628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc4d6a-99d4-485e-b016-b0bd49c1a899 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:46.130842 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 605/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:19:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:19:46.137749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a181ea-851f-4ec4-944c-37fadef18de6 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:46.241966 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a181ea-851f-4ec4-944c-37fadef18de6 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:46.281921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7de448-a79a-41ce-b1fa-3a79c1f7a81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:46.282835 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7de448-a79a-41ce-b1fa-3a79c1f7a81d None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:46.283111 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7de448-a79a-41ce-b1fa-3a79c1f7a81d None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:46.283420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7de448-a79a-41ce-b1fa-3a79c1f7a81d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:46.294107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7de448-a79a-41ce-b1fa-3a79c1f7a81d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:46.295580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7de448-a79a-41ce-b1fa-3a79c1f7a81d None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-ac7de448-a79a-41ce-b1fa-3a79c1f7a81d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:19:46.316341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:46.317499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:46.318618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:19:46.318618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:19:46.318618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:46.318618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:46.318618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 637/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:19:46.324803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc4d6a-99d4-485e-b016-b0bd49c1a899 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:46.431813 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc4d6a-99d4-485e-b016-b0bd49c1a899 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:46.465689 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867db95-a0f5-4521-8747-3e4d0d39325b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:46.466360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867db95-a0f5-4521-8747-3e4d0d39325b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:46.466645 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867db95-a0f5-4521-8747-3e4d0d39325b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:46.466923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867db95-a0f5-4521-8747-3e4d0d39325b None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:46.467669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867db95-a0f5-4521-8747-3e4d0d39325b None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:46.467977 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867db95-a0f5-4521-8747-3e4d0d39325b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:46.469335 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867db95-a0f5-4521-8747-3e4d0d39325b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:46.481388 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 606/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:46.487699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7de448-a79a-41ce-b1fa-3a79c1f7a81d None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:46.608053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7de448-a79a-41ce-b1fa-3a79c1f7a81d None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:46.653029 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de89316-0bac-4678-8f14-b51c31e52186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:46.653882 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de89316-0bac-4678-8f14-b51c31e52186 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:46.654166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de89316-0bac-4678-8f14-b51c31e52186 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:46.654443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de89316-0bac-4678-8f14-b51c31e52186 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:46.661479 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de89316-0bac-4678-8f14-b51c31e52186 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:46.662996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de89316-0bac-4678-8f14-b51c31e52186 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-5de89316-0bac-4678-8f14-b51c31e52186 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:19:46.677969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:46.679013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:46.679973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:19:46.679973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:19:46.679973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:19:46.679973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:19:46.679973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:19:46.679973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:19:46.679973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:19:46.679973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:46.679973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 638/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:19:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:46.686831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867db95-a0f5-4521-8747-3e4d0d39325b None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:46.796929 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867db95-a0f5-4521-8747-3e4d0d39325b None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:46.837098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fa072-0775-47af-a80a-c8682c5e9dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:46.837802 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fa072-0775-47af-a80a-c8682c5e9dcc None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:46.838038 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fa072-0775-47af-a80a-c8682c5e9dcc None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:46.838312 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fa072-0775-47af-a80a-c8682c5e9dcc None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:46.839207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fa072-0775-47af-a80a-c8682c5e9dcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:46.839574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fa072-0775-47af-a80a-c8682c5e9dcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:46.841220 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fa072-0775-47af-a80a-c8682c5e9dcc None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:46.856037 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 607/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:19:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:46.863536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de89316-0bac-4678-8f14-b51c31e52186 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:46.973565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de89316-0bac-4678-8f14-b51c31e52186 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:47.015965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69052144-130a-4ace-935a-40a022f2c694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:47.016829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052144-130a-4ace-935a-40a022f2c694 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:47.017320 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052144-130a-4ace-935a-40a022f2c694 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:47.017748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052144-130a-4ace-935a-40a022f2c694 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:47.019312 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69052144-130a-4ace-935a-40a022f2c694 None admin] RBAC: Adding query filter params (role.id=7b2e521b45dc4a98adad996ca0027de6, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=f747dcb955fa499a972c46587d3db294) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:47.019854 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69052144-130a-4ace-935a-40a022f2c694 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:47.021723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69052144-130a-4ace-935a-40a022f2c694 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:47.031093 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 639/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:19:46 2022] GET /identity/v3/role_assignments?user.id=f747dcb955fa499a972c46587d3db294&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=7b2e521b45dc4a98adad996ca0027de6 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:49.741635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76a220-1bd7-4548-a5c2-648993442a23 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:49.742225 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76a220-1bd7-4548-a5c2-648993442a23 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:49.742432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76a220-1bd7-4548-a5c2-648993442a23 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:49.744035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 608/1247] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:19:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:19:49.750110 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27b1e1-45c0-4d88-8cac-2f29abcd385a None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:49.750558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27b1e1-45c0-4d88-8cac-2f29abcd385a None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:49.750651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27b1e1-45c0-4d88-8cac-2f29abcd385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:49.769179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4e27b1e1-45c0-4d88-8cac-2f29abcd385a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:19:49.770706 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4e27b1e1-45c0-4d88-8cac-2f29abcd385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:19:49.845212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 640/1248] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:19:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 18:19:49.870903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76a220-1bd7-4548-a5c2-648993442a23 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:49.981041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76a220-1bd7-4548-a5c2-648993442a23 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:50.017912 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742864d-9830-446c-ab86-7b863ab08a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:50.019254 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742864d-9830-446c-ab86-7b863ab08a2e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:50.019254 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742864d-9830-446c-ab86-7b863ab08a2e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:50.019254 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742864d-9830-446c-ab86-7b863ab08a2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:50.030327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742864d-9830-446c-ab86-7b863ab08a2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-7742864d-9830-446c-ab86-7b863ab08a2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:19:50.037964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:50.038963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:50.040635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:50.040635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:50.040635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:50.040635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:50.040635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:19:50.040635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:19:50.040635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:19:50.040635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:19:50.040635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:19:50.040635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:19:50.040635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:19:50.040635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:50.040635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 609/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:19:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:50.045863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27b1e1-45c0-4d88-8cac-2f29abcd385a None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:50.155793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27b1e1-45c0-4d88-8cac-2f29abcd385a None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:50.192426 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb1ed2-4c9c-4423-8296-b5a2778df6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:50.193152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb1ed2-4c9c-4423-8296-b5a2778df6af None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:50.193405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb1ed2-4c9c-4423-8296-b5a2778df6af None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:50.193680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb1ed2-4c9c-4423-8296-b5a2778df6af None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:50.194420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb1ed2-4c9c-4423-8296-b5a2778df6af None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:50.194720 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb1ed2-4c9c-4423-8296-b5a2778df6af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:50.196627 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb1ed2-4c9c-4423-8296-b5a2778df6af None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:50.204885 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 641/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:19:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:19:50.210680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742864d-9830-446c-ab86-7b863ab08a2e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:50.326180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742864d-9830-446c-ab86-7b863ab08a2e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:50.361333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c033bd-c44a-4258-a26c-7c4c29df7a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:50.362118 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c033bd-c44a-4258-a26c-7c4c29df7a7c None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:50.362400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c033bd-c44a-4258-a26c-7c4c29df7a7c None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:50.362675 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c033bd-c44a-4258-a26c-7c4c29df7a7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:50.374017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c033bd-c44a-4258-a26c-7c4c29df7a7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:50.375488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c033bd-c44a-4258-a26c-7c4c29df7a7c None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-21c033bd-c44a-4258-a26c-7c4c29df7a7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:19:50.392804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:50.393790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:50.394697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:19:50.394697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:19:50.394697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:50.394697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:50.394697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 610/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:19:50.401111 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb1ed2-4c9c-4423-8296-b5a2778df6af None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:50.505969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb1ed2-4c9c-4423-8296-b5a2778df6af None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:50.542093 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fdae3-dab5-4cb5-a4b0-351e4e491b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:50.542761 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fdae3-dab5-4cb5-a4b0-351e4e491b13 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:50.543042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fdae3-dab5-4cb5-a4b0-351e4e491b13 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:50.543405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fdae3-dab5-4cb5-a4b0-351e4e491b13 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:50.544257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fdae3-dab5-4cb5-a4b0-351e4e491b13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:50.544567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fdae3-dab5-4cb5-a4b0-351e4e491b13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:50.546135 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fdae3-dab5-4cb5-a4b0-351e4e491b13 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:50.563392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 642/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:19:50.564541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c033bd-c44a-4258-a26c-7c4c29df7a7c None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:50.675738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c033bd-c44a-4258-a26c-7c4c29df7a7c None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:50.709498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aac8ba-ff3d-47fc-bb49-53b2da86cbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:50.710387 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aac8ba-ff3d-47fc-bb49-53b2da86cbe2 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:50.710743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aac8ba-ff3d-47fc-bb49-53b2da86cbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:50.710958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aac8ba-ff3d-47fc-bb49-53b2da86cbe2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:50.719967 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aac8ba-ff3d-47fc-bb49-53b2da86cbe2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:50.721615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aac8ba-ff3d-47fc-bb49-53b2da86cbe2 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-50aac8ba-ff3d-47fc-bb49-53b2da86cbe2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:19:50.737592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:50.738605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:50.739776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:50.739776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:50.739776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:19:50.739776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:19:50.739776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:19:50.739776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:19:50.739776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:19:50.739776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:19:50.739776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 18:19:50.739776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:50.739776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 611/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:50.745058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fdae3-dab5-4cb5-a4b0-351e4e491b13 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:50.849299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fdae3-dab5-4cb5-a4b0-351e4e491b13 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:50.884535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d56ecc-58fa-4a32-af58-42b99d9726df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:50.885201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d56ecc-58fa-4a32-af58-42b99d9726df None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:50.885546 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d56ecc-58fa-4a32-af58-42b99d9726df None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:50.885847 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d56ecc-58fa-4a32-af58-42b99d9726df None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:50.886828 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d56ecc-58fa-4a32-af58-42b99d9726df None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:50.887247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d56ecc-58fa-4a32-af58-42b99d9726df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:50.888703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d56ecc-58fa-4a32-af58-42b99d9726df None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:50.912150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 643/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Mon May 9 18:19:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:50.914809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aac8ba-ff3d-47fc-bb49-53b2da86cbe2 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:51.023288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aac8ba-ff3d-47fc-bb49-53b2da86cbe2 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:51.059686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6a911-7c99-4fc2-bbd3-f39b679a0e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:51.060367 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6a911-7c99-4fc2-bbd3-f39b679a0e5e None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:51.060651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6a911-7c99-4fc2-bbd3-f39b679a0e5e None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:51.060930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6a911-7c99-4fc2-bbd3-f39b679a0e5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:51.130692 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6a911-7c99-4fc2-bbd3-f39b679a0e5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:51.132207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6a911-7c99-4fc2-bbd3-f39b679a0e5e None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:51.232218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 612/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:19:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:19:51.237869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d56ecc-58fa-4a32-af58-42b99d9726df None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:51.338044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d56ecc-58fa-4a32-af58-42b99d9726df None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:51.371774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56f1f2-4236-4d00-a51b-732103dd4a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:51.372573 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56f1f2-4236-4d00-a51b-732103dd4a66 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:51.372848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56f1f2-4236-4d00-a51b-732103dd4a66 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:51.373116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56f1f2-4236-4d00-a51b-732103dd4a66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:51.384457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56f1f2-4236-4d00-a51b-732103dd4a66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:51.385913 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56f1f2-4236-4d00-a51b-732103dd4a66 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-0c56f1f2-4236-4d00-a51b-732103dd4a66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:19:51.403541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:51.404737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:51.406735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:19:51.406735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:19:51.406735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:51.406735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:51.406735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 644/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:51.410597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6a911-7c99-4fc2-bbd3-f39b679a0e5e None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:51.507941 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6a911-7c99-4fc2-bbd3-f39b679a0e5e None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:51.541047 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63424d6-ade9-4381-93d7-d6cd03a5ad52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:51.541712 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63424d6-ade9-4381-93d7-d6cd03a5ad52 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:51.541995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63424d6-ade9-4381-93d7-d6cd03a5ad52 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:51.542270 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63424d6-ade9-4381-93d7-d6cd03a5ad52 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:51.542978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63424d6-ade9-4381-93d7-d6cd03a5ad52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:51.543321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63424d6-ade9-4381-93d7-d6cd03a5ad52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:51.544669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63424d6-ade9-4381-93d7-d6cd03a5ad52 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:51.556323 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 613/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:51.562174 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56f1f2-4236-4d00-a51b-732103dd4a66 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:51.673041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56f1f2-4236-4d00-a51b-732103dd4a66 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:51.704878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591b602-a786-4e44-9eef-f65e5837d37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:51.705662 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591b602-a786-4e44-9eef-f65e5837d37f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:51.705945 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591b602-a786-4e44-9eef-f65e5837d37f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:51.706218 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591b602-a786-4e44-9eef-f65e5837d37f None admin] PATH_INFO: `/v3/projects/service` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:51.712629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591b602-a786-4e44-9eef-f65e5837d37f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:51.714052 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591b602-a786-4e44-9eef-f65e5837d37f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-8591b602-a786-4e44-9eef-f65e5837d37f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:19:51.727910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:51.728993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:51.730065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:19:51.730065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:19:51.730065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:19:51.730065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:19:51.730065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:19:51.730065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:19:51.730065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:19:51.730065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:51.730065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 645/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:19:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:51.734836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63424d6-ade9-4381-93d7-d6cd03a5ad52 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:51.828456 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63424d6-ade9-4381-93d7-d6cd03a5ad52 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:51.859518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfecbbea-2619-42a4-9cda-80dd8024cd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:51.860165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecbbea-2619-42a4-9cda-80dd8024cd0b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:51.860448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecbbea-2619-42a4-9cda-80dd8024cd0b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:51.860724 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecbbea-2619-42a4-9cda-80dd8024cd0b None admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:51.861495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfecbbea-2619-42a4-9cda-80dd8024cd0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:51.861803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfecbbea-2619-42a4-9cda-80dd8024cd0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:51.863237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfecbbea-2619-42a4-9cda-80dd8024cd0b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:51.875306 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 614/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:19:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:51.881059 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591b602-a786-4e44-9eef-f65e5837d37f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:51.973807 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591b602-a786-4e44-9eef-f65e5837d37f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:52.005474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb3ab5-8bc0-4d18-aae6-01e2c062537b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:52.006107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb3ab5-8bc0-4d18-aae6-01e2c062537b None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:52.006386 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb3ab5-8bc0-4d18-aae6-01e2c062537b None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:52.006655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb3ab5-8bc0-4d18-aae6-01e2c062537b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:52.007490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb3ab5-8bc0-4d18-aae6-01e2c062537b None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=f747dcb955fa499a972c46587d3db294) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:52.007796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb3ab5-8bc0-4d18-aae6-01e2c062537b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:52.009338 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb3ab5-8bc0-4d18-aae6-01e2c062537b None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:52.014555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 646/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:19:51 2022] GET /identity/v3/role_assignments?user.id=f747dcb955fa499a972c46587d3db294&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 259 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:54.196991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a02f64-4287-4258-b47e-7955164832af None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:54.197540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a02f64-4287-4258-b47e-7955164832af None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:54.197540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a02f64-4287-4258-b47e-7955164832af None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:54.198896 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 615/1261] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:19:54.203969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33587ae7-904f-4f6a-a226-ae85e294d8d1 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:54.204366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33587ae7-904f-4f6a-a226-ae85e294d8d1 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:54.204514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33587ae7-904f-4f6a-a226-ae85e294d8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:54.221280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-33587ae7-904f-4f6a-a226-ae85e294d8d1 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:19:54.222382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-33587ae7-904f-4f6a-a226-ae85e294d8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:19:54.284273 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 647/1262] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:19:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:19:54.306618 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a02f64-4287-4258-b47e-7955164832af None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:54.401492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a02f64-4287-4258-b47e-7955164832af None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:54.432918 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83b6f4-fbce-4723-9c33-f19cd8544523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:54.433695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83b6f4-fbce-4723-9c33-f19cd8544523 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:54.433974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83b6f4-fbce-4723-9c33-f19cd8544523 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:54.434246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83b6f4-fbce-4723-9c33-f19cd8544523 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:54.443632 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83b6f4-fbce-4723-9c33-f19cd8544523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-7c83b6f4-fbce-4723-9c33-f19cd8544523 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:19:54.452073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:54.453186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:54.454179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:54.454179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:54.454179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:54.454179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:54.454179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:19:54.454179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:19:54.454179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:19:54.454179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:19:54.454179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:19:54.454179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:19:54.454179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:19:54.454179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:54.454179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 616/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:19:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:54.459201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33587ae7-904f-4f6a-a226-ae85e294d8d1 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:54.556415 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33587ae7-904f-4f6a-a226-ae85e294d8d1 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:54.592696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a39a8b-9128-4ac9-9c05-31d2c87ebc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:54.593362 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a39a8b-9128-4ac9-9c05-31d2c87ebc55 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:54.593640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a39a8b-9128-4ac9-9c05-31d2c87ebc55 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:54.593955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a39a8b-9128-4ac9-9c05-31d2c87ebc55 None admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:54.594742 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a39a8b-9128-4ac9-9c05-31d2c87ebc55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:54.595032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a39a8b-9128-4ac9-9c05-31d2c87ebc55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:54.596574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a39a8b-9128-4ac9-9c05-31d2c87ebc55 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:54.606550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 648/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:19:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:54.612784 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83b6f4-fbce-4723-9c33-f19cd8544523 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:54.711428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83b6f4-fbce-4723-9c33-f19cd8544523 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:54.742343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3f264-1b3e-4a05-8b47-7b1fd79ed4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:54.743153 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3f264-1b3e-4a05-8b47-7b1fd79ed4a7 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:54.743463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3f264-1b3e-4a05-8b47-7b1fd79ed4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:54.743738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3f264-1b3e-4a05-8b47-7b1fd79ed4a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:54.753844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3f264-1b3e-4a05-8b47-7b1fd79ed4a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:54.755295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3f264-1b3e-4a05-8b47-7b1fd79ed4a7 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-caa3f264-1b3e-4a05-8b47-7b1fd79ed4a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:19:54.777298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:54.778370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:54.779514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:19:54.779514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:19:54.779514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:54.779514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:54.779514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 617/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:54.784184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a39a8b-9128-4ac9-9c05-31d2c87ebc55 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:54.876340 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a39a8b-9128-4ac9-9c05-31d2c87ebc55 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:54.907346 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b77ef7-fbcc-468f-82d8-b58d2fa1074b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:54.907978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b77ef7-fbcc-468f-82d8-b58d2fa1074b None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:54.908253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b77ef7-fbcc-468f-82d8-b58d2fa1074b None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:54.908521 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b77ef7-fbcc-468f-82d8-b58d2fa1074b None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:54.909214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b77ef7-fbcc-468f-82d8-b58d2fa1074b None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:54.909510 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b77ef7-fbcc-468f-82d8-b58d2fa1074b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:54.910828 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b77ef7-fbcc-468f-82d8-b58d2fa1074b None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:54.921731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 649/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:54.927222 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3f264-1b3e-4a05-8b47-7b1fd79ed4a7 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:55.020943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3f264-1b3e-4a05-8b47-7b1fd79ed4a7 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:55.052691 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41026b8e-58da-498d-8098-d5d33b5cd47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:55.053539 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41026b8e-58da-498d-8098-d5d33b5cd47f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:55.053833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41026b8e-58da-498d-8098-d5d33b5cd47f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:55.054332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41026b8e-58da-498d-8098-d5d33b5cd47f None admin] PATH_INFO: `/v3/users/placement` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:55.063847 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41026b8e-58da-498d-8098-d5d33b5cd47f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:55.065269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41026b8e-58da-498d-8098-d5d33b5cd47f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-41026b8e-58da-498d-8098-d5d33b5cd47f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:19:55.079473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:55.080585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:55.081718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:55.081718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:55.081718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:19:55.081718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:19:55.081718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:19:55.081718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:19:55.081718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:19:55.081718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:19:55.081718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 18:19:55.081718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:55.081718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 618/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:55.086248 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b77ef7-fbcc-468f-82d8-b58d2fa1074b None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:55.178322 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b77ef7-fbcc-468f-82d8-b58d2fa1074b None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:55.210803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541b88e-81a1-4f82-a7a6-2d60ba9738c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:55.211482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541b88e-81a1-4f82-a7a6-2d60ba9738c4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:55.211756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541b88e-81a1-4f82-a7a6-2d60ba9738c4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:55.212028 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541b88e-81a1-4f82-a7a6-2d60ba9738c4 None admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:55.212833 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541b88e-81a1-4f82-a7a6-2d60ba9738c4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:55.213130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541b88e-81a1-4f82-a7a6-2d60ba9738c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:55.214521 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541b88e-81a1-4f82-a7a6-2d60ba9738c4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:55.230315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 650/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Mon May 9 18:19:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:55.235910 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41026b8e-58da-498d-8098-d5d33b5cd47f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:55.330012 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41026b8e-58da-498d-8098-d5d33b5cd47f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:55.365200 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309ae68-6d8c-46f4-bd20-25a6d3640070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:55.365993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309ae68-6d8c-46f4-bd20-25a6d3640070 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:55.366274 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309ae68-6d8c-46f4-bd20-25a6d3640070 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:55.366548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309ae68-6d8c-46f4-bd20-25a6d3640070 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:55.377558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309ae68-6d8c-46f4-bd20-25a6d3640070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:55.378991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309ae68-6d8c-46f4-bd20-25a6d3640070 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-7309ae68-6d8c-46f4-bd20-25a6d3640070 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:19:55.395224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:55.396242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:55.397332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:19:55.397332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:19:55.397332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:55.397332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:55.397332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 619/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:55.402190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541b88e-81a1-4f82-a7a6-2d60ba9738c4 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:55.493496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541b88e-81a1-4f82-a7a6-2d60ba9738c4 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:55.524669 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c09060-87f8-4797-8db2-f690577a1271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:55.525305 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c09060-87f8-4797-8db2-f690577a1271 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:55.525580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c09060-87f8-4797-8db2-f690577a1271 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:55.525847 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c09060-87f8-4797-8db2-f690577a1271 None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:55.526539 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c09060-87f8-4797-8db2-f690577a1271 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:55.526839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c09060-87f8-4797-8db2-f690577a1271 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:55.528228 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c09060-87f8-4797-8db2-f690577a1271 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:55.538665 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 651/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:55.544563 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309ae68-6d8c-46f4-bd20-25a6d3640070 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:55.649157 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309ae68-6d8c-46f4-bd20-25a6d3640070 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:55.679827 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed1775-05d0-4978-ae41-258a9ac6a2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:55.681225 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed1775-05d0-4978-ae41-258a9ac6a2f5 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:55.681498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed1775-05d0-4978-ae41-258a9ac6a2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:55.681764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed1775-05d0-4978-ae41-258a9ac6a2f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:55.687972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed1775-05d0-4978-ae41-258a9ac6a2f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:55.689384 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed1775-05d0-4978-ae41-258a9ac6a2f5 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-6aed1775-05d0-4978-ae41-258a9ac6a2f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:19:55.702837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:55.703901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:55.705003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:19:55.705003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:19:55.705003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:19:55.705003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:19:55.705003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:19:55.705003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:19:55.705003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:19:55.705003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:55.705003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 620/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:19:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:19:55.709745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c09060-87f8-4797-8db2-f690577a1271 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:55.805425 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c09060-87f8-4797-8db2-f690577a1271 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:55.835474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54956a9-6827-44b0-9932-e9b4c7b409c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:55.836630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54956a9-6827-44b0-9932-e9b4c7b409c0 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:55.836912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54956a9-6827-44b0-9932-e9b4c7b409c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:55.837182 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54956a9-6827-44b0-9932-e9b4c7b409c0 None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:55.837941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54956a9-6827-44b0-9932-e9b4c7b409c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:55.838244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54956a9-6827-44b0-9932-e9b4c7b409c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:55.839673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54956a9-6827-44b0-9932-e9b4c7b409c0 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:55.850771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 652/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:19:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:55.857589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed1775-05d0-4978-ae41-258a9ac6a2f5 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:55.949483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed1775-05d0-4978-ae41-258a9ac6a2f5 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:55.981775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c5786-9664-447e-a96b-908948fd1b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:55.982652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c5786-9664-447e-a96b-908948fd1b7a None admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:55.982934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c5786-9664-447e-a96b-908948fd1b7a None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:55.983236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c5786-9664-447e-a96b-908948fd1b7a None admin] PATH_INFO: `/v3/projects/4849b58351c74caf92a584686c74e397/users/f747dcb955fa499a972c46587d3db294/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:55.985129 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c5786-9664-447e-a96b-908948fd1b7a None admin] RBAC: Authorizing `identity:create_grant(project_id=4849b58351c74caf92a584686c74e397, user_id=f747dcb955fa499a972c46587d3db294, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:55.986536 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c5786-9664-447e-a96b-908948fd1b7a None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:55.998899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 621/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Mon May 9 18:19:55 2022] PUT /identity/v3/projects/4849b58351c74caf92a584686c74e397/users/f747dcb955fa499a972c46587d3db294/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:19:58.174857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeac019-1b9d-47cd-accd-98cbb0ee4e04 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:58.175439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeac019-1b9d-47cd-accd-98cbb0ee4e04 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:58.175439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeac019-1b9d-47cd-accd-98cbb0ee4e04 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:58.176790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 653/1274] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:19:58.182471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d846da-0749-4db2-8135-82b1e0b03eed None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:58.182862 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d846da-0749-4db2-8135-82b1e0b03eed None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:58.183024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d846da-0749-4db2-8135-82b1e0b03eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:58.199561 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-a8d846da-0749-4db2-8135-82b1e0b03eed None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:19:58.200721 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a8d846da-0749-4db2-8135-82b1e0b03eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:19:58.263088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 622/1275] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:19:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:19:58.286400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeac019-1b9d-47cd-accd-98cbb0ee4e04 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:58.381705 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeac019-1b9d-47cd-accd-98cbb0ee4e04 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:58.414287 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320079fd-dbcc-49ed-94dd-e45f892e3809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:58.415073 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320079fd-dbcc-49ed-94dd-e45f892e3809 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:58.415383 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320079fd-dbcc-49ed-94dd-e45f892e3809 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:58.415657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320079fd-dbcc-49ed-94dd-e45f892e3809 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:58.425193 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320079fd-dbcc-49ed-94dd-e45f892e3809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-320079fd-dbcc-49ed-94dd-e45f892e3809 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 18:19:58.432777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:58.433841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:58.434721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:58.434721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:58.434721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:58.434721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:58.434721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 18:19:58.434721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 18:19:58.434721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 18:19:58.434721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 18:19:58.434721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 18:19:58.434721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 18:19:58.434721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 18:19:58.434721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:58.434721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 654/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Mon May 9 18:19:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:58.439850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d846da-0749-4db2-8135-82b1e0b03eed None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:58.531934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d846da-0749-4db2-8135-82b1e0b03eed None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:58.567985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3e71e-375b-466a-9843-b17130b55ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:58.568742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3e71e-375b-466a-9843-b17130b55ffc None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:58.569034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3e71e-375b-466a-9843-b17130b55ffc None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:58.569432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3e71e-375b-466a-9843-b17130b55ffc None admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:58.570208 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3e71e-375b-466a-9843-b17130b55ffc None admin] RBAC: Adding query filter params (name=admin) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:58.571114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3e71e-375b-466a-9843-b17130b55ffc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:58.572691 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3e71e-375b-466a-9843-b17130b55ffc None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:58.581888 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 623/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Mon May 9 18:19:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:58.588499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320079fd-dbcc-49ed-94dd-e45f892e3809 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:58.693958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320079fd-dbcc-49ed-94dd-e45f892e3809 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:58.725199 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd20f5-5a1a-446d-a3d8-1af4b95cfadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:58.725972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd20f5-5a1a-446d-a3d8-1af4b95cfadd None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:58.726247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd20f5-5a1a-446d-a3d8-1af4b95cfadd None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:58.726535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd20f5-5a1a-446d-a3d8-1af4b95cfadd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:58.736215 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd20f5-5a1a-446d-a3d8-1af4b95cfadd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:58.737626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd20f5-5a1a-446d-a3d8-1af4b95cfadd None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-7bdd20f5-5a1a-446d-a3d8-1af4b95cfadd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:19:58.754401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:58.755472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:58.761684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:19:58.761684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:19:58.761684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:58.761684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:58.761684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 655/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 18:19:58.762061 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3e71e-375b-466a-9843-b17130b55ffc None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:58.855385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3e71e-375b-466a-9843-b17130b55ffc None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:58.887329 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebadbfa6-ccdd-4432-961d-7b0b5ffff5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:58.887973 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadbfa6-ccdd-4432-961d-7b0b5ffff5f8 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:58.888255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadbfa6-ccdd-4432-961d-7b0b5ffff5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:58.888529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadbfa6-ccdd-4432-961d-7b0b5ffff5f8 None admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:58.889232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebadbfa6-ccdd-4432-961d-7b0b5ffff5f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:58.889535 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebadbfa6-ccdd-4432-961d-7b0b5ffff5f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:58.890880 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebadbfa6-ccdd-4432-961d-7b0b5ffff5f8 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:58.901940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 624/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:19:58.907465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd20f5-5a1a-446d-a3d8-1af4b95cfadd None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:58.998573 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd20f5-5a1a-446d-a3d8-1af4b95cfadd None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:59.029250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6568d69-552f-41b7-814c-45cc382e8fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:59.030028 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6568d69-552f-41b7-814c-45cc382e8fd5 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:59.030303 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6568d69-552f-41b7-814c-45cc382e8fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:59.030572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6568d69-552f-41b7-814c-45cc382e8fd5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:59.037781 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6568d69-552f-41b7-814c-45cc382e8fd5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:59.039238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6568d69-552f-41b7-814c-45cc382e8fd5 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-f6568d69-552f-41b7-814c-45cc382e8fd5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 18:19:59.052661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:59.053737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:59.055407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:59.055407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:59.055407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 18:19:59.055407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 18:19:59.055407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 18:19:59.055407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 18:19:59.055407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 18:19:59.055407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 18:19:59.055407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 18:19:59.055407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:19:59.055407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 656/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:59.061937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebadbfa6-ccdd-4432-961d-7b0b5ffff5f8 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:59.154588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebadbfa6-ccdd-4432-961d-7b0b5ffff5f8 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:59.186503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f10b1-0123-42c1-8a37-90ec3698cd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:59.187155 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f10b1-0123-42c1-8a37-90ec3698cd53 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:59.187470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f10b1-0123-42c1-8a37-90ec3698cd53 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:59.187747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f10b1-0123-42c1-8a37-90ec3698cd53 None admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:59.188564 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f10b1-0123-42c1-8a37-90ec3698cd53 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:59.188871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f10b1-0123-42c1-8a37-90ec3698cd53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:59.190258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f10b1-0123-42c1-8a37-90ec3698cd53 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:59.206198 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 625/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Mon May 9 18:19:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:59.211686 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6568d69-552f-41b7-814c-45cc382e8fd5 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:59.304439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6568d69-552f-41b7-814c-45cc382e8fd5 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:59.335011 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61881520-b490-4b0a-b304-86168d47716e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:59.335655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61881520-b490-4b0a-b304-86168d47716e None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:59.335967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61881520-b490-4b0a-b304-86168d47716e None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:59.336239 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61881520-b490-4b0a-b304-86168d47716e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:59.397341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61881520-b490-4b0a-b304-86168d47716e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:59.399069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61881520-b490-4b0a-b304-86168d47716e None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:59.490726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 657/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Mon May 9 18:19:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 18:19:59.496374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f10b1-0123-42c1-8a37-90ec3698cd53 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:59.592913 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f10b1-0123-42c1-8a37-90ec3698cd53 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:59.625321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722bf6d-beee-4fa2-b00f-83575d8e017f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:59.626098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722bf6d-beee-4fa2-b00f-83575d8e017f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:59.626378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722bf6d-beee-4fa2-b00f-83575d8e017f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:59.626657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722bf6d-beee-4fa2-b00f-83575d8e017f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:59.636139 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722bf6d-beee-4fa2-b00f-83575d8e017f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:59.637553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722bf6d-beee-4fa2-b00f-83575d8e017f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-2722bf6d-beee-4fa2-b00f-83575d8e017f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 18:19:59.654509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:59.655568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:59.656666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 18:19:59.656666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 18:19:59.656666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 18:19:59.656666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:59.656666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 626/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:59.661449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61881520-b490-4b0a-b304-86168d47716e None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:59.753804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61881520-b490-4b0a-b304-86168d47716e None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:59.785937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4e276-205b-4e85-8875-595161498f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:59.786578 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4e276-205b-4e85-8875-595161498f9f None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:59.786859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4e276-205b-4e85-8875-595161498f9f None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:59.787136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4e276-205b-4e85-8875-595161498f9f None admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:59.787876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4e276-205b-4e85-8875-595161498f9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:19:59.788178 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4e276-205b-4e85-8875-595161498f9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:59.790054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4e276-205b-4e85-8875-595161498f9f None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:59.800548 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 658/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Mon May 9 18:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:19:59.806132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722bf6d-beee-4fa2-b00f-83575d8e017f None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:19:59.897673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722bf6d-beee-4fa2-b00f-83575d8e017f None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:19:59.928929 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095154c-cd41-4a7f-9d51-e4a06c4b7ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:19:59.929727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095154c-cd41-4a7f-9d51-e4a06c4b7ef5 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:19:59.930006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095154c-cd41-4a7f-9d51-e4a06c4b7ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:19:59.930277 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095154c-cd41-4a7f-9d51-e4a06c4b7ef5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:19:59.936478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095154c-cd41-4a7f-9d51-e4a06c4b7ef5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:19:59.937868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095154c-cd41-4a7f-9d51-e4a06c4b7ef5 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application [None req-d095154c-cd41-4a7f-9d51-e4a06c4b7ef5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 18:19:59.951331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application with Lock( May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._enter() May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.creator() May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:19:59.952390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application created_value = creator( May 09 18:19:59.953490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 18:19:59.953490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 18:19:59.953490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 18:19:59.953490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 18:19:59.953490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 18:19:59.953490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 18:19:59.953490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 18:19:59.953490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: ERROR keystone.server.flask.application May 09 18:19:59.953490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 627/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Mon May 9 18:19:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:19:59.958040 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4e276-205b-4e85-8875-595161498f9f None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:20:00.051550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4e276-205b-4e85-8875-595161498f9f None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:20:00.082309 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7318a7-4ed7-4aac-b7f1-53418539a54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:20:00.082953 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7318a7-4ed7-4aac-b7f1-53418539a54d None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:00.083260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7318a7-4ed7-4aac-b7f1-53418539a54d None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:00.083539 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7318a7-4ed7-4aac-b7f1-53418539a54d None admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:00.084308 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7318a7-4ed7-4aac-b7f1-53418539a54d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:20:00.084614 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7318a7-4ed7-4aac-b7f1-53418539a54d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:20:00.086558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7318a7-4ed7-4aac-b7f1-53418539a54d None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:20:00.097842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 659/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Mon May 9 18:19:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:20:00.103841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095154c-cd41-4a7f-9d51-e4a06c4b7ef5 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:20:00.197962 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095154c-cd41-4a7f-9d51-e4a06c4b7ef5 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:20:00.229982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789d260-1063-4073-920a-b8de2299e387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:20:00.230622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789d260-1063-4073-920a-b8de2299e387 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:00.230903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789d260-1063-4073-920a-b8de2299e387 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:00.231200 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789d260-1063-4073-920a-b8de2299e387 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:00.232014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789d260-1063-4073-920a-b8de2299e387 None admin] RBAC: Adding query filter params (role.id=ee562e637fc64608b9129ddd09ec0cdc, scope.project.id=4849b58351c74caf92a584686c74e397, user.id=f747dcb955fa499a972c46587d3db294) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:20:00.232320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789d260-1063-4073-920a-b8de2299e387 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:20:00.233764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789d260-1063-4073-920a-b8de2299e387 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:20:00.239347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 628/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Mon May 9 18:20:00 2022] GET /identity/v3/role_assignments?user.id=f747dcb955fa499a972c46587d3db294&scope.project.id=4849b58351c74caf92a584686c74e397&role.id=ee562e637fc64608b9129ddd09ec0cdc => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:20:02.440797 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322bb77-59e5-40ef-8683-27c9c3b3a419 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:02.441325 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322bb77-59e5-40ef-8683-27c9c3b3a419 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:02.441325 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322bb77-59e5-40ef-8683-27c9c3b3a419 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:02.442694 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 660/1288] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:20:02.449764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26aa22a-c25f-443f-8471-e8a62d3997c4 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:02.449764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26aa22a-c25f-443f-8471-e8a62d3997c4 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:02.449764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26aa22a-c25f-443f-8471-e8a62d3997c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:02.467625 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-b26aa22a-c25f-443f-8471-e8a62d3997c4 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:20:02.468787 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b26aa22a-c25f-443f-8471-e8a62d3997c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:20:02.530401 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 629/1289] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:20:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:20:02.552754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322bb77-59e5-40ef-8683-27c9c3b3a419 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:20:02.656180 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322bb77-59e5-40ef-8683-27c9c3b3a419 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:20:02.687884 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0301540-8521-42a3-bfc6-6666b851d888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:20:02.688666 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0301540-8521-42a3-bfc6-6666b851d888 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:02.688945 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0301540-8521-42a3-bfc6-6666b851d888 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:02.689214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0301540-8521-42a3-bfc6-6666b851d888 None admin] PATH_INFO: `/v3/services/placement` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:02.689850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0301540-8521-42a3-bfc6-6666b851d888 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:20:02.691271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0301540-8521-42a3-bfc6-6666b851d888 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-b0301540-8521-42a3-bfc6-6666b851d888 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:20:02.703119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:20:02.704350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:20:02.705397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 661/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Mon May 9 18:20:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:20:02.710399 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26aa22a-c25f-443f-8471-e8a62d3997c4 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:20:02.801288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26aa22a-c25f-443f-8471-e8a62d3997c4 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:20:02.832947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d4b0f-6d02-4184-9f44-b830a4bbf01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:20:02.833589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d4b0f-6d02-4184-9f44-b830a4bbf01b None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:02.833866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d4b0f-6d02-4184-9f44-b830a4bbf01b None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:02.834137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d4b0f-6d02-4184-9f44-b830a4bbf01b None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:02.834817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d4b0f-6d02-4184-9f44-b830a4bbf01b None admin] RBAC: Adding query filter params (name=placement) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:20:02.835115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d4b0f-6d02-4184-9f44-b830a4bbf01b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:20:02.836515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d4b0f-6d02-4184-9f44-b830a4bbf01b None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:20:02.841013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 630/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Mon May 9 18:20:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:20:02.846731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0301540-8521-42a3-bfc6-6666b851d888 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:20:02.939563 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0301540-8521-42a3-bfc6-6666b851d888 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:20:02.970142 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4a4cb-e22e-4350-852e-9f2311812db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:20:02.970790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4a4cb-e22e-4350-852e-9f2311812db4 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:02.971069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4a4cb-e22e-4350-852e-9f2311812db4 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:02.971369 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4a4cb-e22e-4350-852e-9f2311812db4 None admin] PATH_INFO: `/v3/services` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:02.972058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4a4cb-e22e-4350-852e-9f2311812db4 None admin] RBAC: Adding query filter params (type=placement) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:20:02.972356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4a4cb-e22e-4350-852e-9f2311812db4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:20:02.973699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4a4cb-e22e-4350-852e-9f2311812db4 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:20:02.978457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 662/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Mon May 9 18:20:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:20:05.146616 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d846cf4-267c-471d-bd3e-ca29400c841d None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:05.147163 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d846cf4-267c-471d-bd3e-ca29400c841d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:05.147163 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d846cf4-267c-471d-bd3e-ca29400c841d None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:05.149176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 631/1293] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:20:05.154353 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739f11-40d0-49d9-9a74-dbda76b13900 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:05.154774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739f11-40d0-49d9-9a74-dbda76b13900 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:05.154902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739f11-40d0-49d9-9a74-dbda76b13900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:05.170964 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-79739f11-40d0-49d9-9a74-dbda76b13900 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:20:05.172156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-79739f11-40d0-49d9-9a74-dbda76b13900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:20:05.234157 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 663/1294] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:20:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:20:05.256276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d846cf4-267c-471d-bd3e-ca29400c841d None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:20:05.352878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d846cf4-267c-471d-bd3e-ca29400c841d None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:20:05.384604 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d966580-49d6-49fc-9d9b-c869597fc2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:20:05.385254 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d966580-49d6-49fc-9d9b-c869597fc2a2 None admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:05.385537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d966580-49d6-49fc-9d9b-c869597fc2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:05.385810 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d966580-49d6-49fc-9d9b-c869597fc2a2 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:05.386519 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d966580-49d6-49fc-9d9b-c869597fc2a2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:20:05.387888 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d966580-49d6-49fc-9d9b-c869597fc2a2 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:20:05.400854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 632/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Mon May 9 18:20:05 2022] POST /identity/v3/services => generated 246 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:20:07.605916 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c08457-741e-495a-9e27-d0adf1bb1471 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:07.606468 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c08457-741e-495a-9e27-d0adf1bb1471 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:07.606468 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c08457-741e-495a-9e27-d0adf1bb1471 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:07.607845 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 664/1296] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:20:07.613148 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776ebff-9d43-4e14-8bf6-33f018734f14 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:07.613624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776ebff-9d43-4e14-8bf6-33f018734f14 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:07.613735 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776ebff-9d43-4e14-8bf6-33f018734f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:07.630870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-a776ebff-9d43-4e14-8bf6-33f018734f14 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:20:07.632073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a776ebff-9d43-4e14-8bf6-33f018734f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:20:07.699946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 633/1297] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:20:07 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:20:07.722518 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c08457-741e-495a-9e27-d0adf1bb1471 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:20:07.815565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c08457-741e-495a-9e27-d0adf1bb1471 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:20:07.846604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297d504-53ed-4ed5-89b2-43bba473dbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:20:07.847398 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297d504-53ed-4ed5-89b2-43bba473dbda None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:07.847678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297d504-53ed-4ed5-89b2-43bba473dbda None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:07.847947 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297d504-53ed-4ed5-89b2-43bba473dbda None admin] PATH_INFO: `/v3/services/placement` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:07.848577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297d504-53ed-4ed5-89b2-43bba473dbda None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:20:07.849911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297d504-53ed-4ed5-89b2-43bba473dbda None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-9297d504-53ed-4ed5-89b2-43bba473dbda None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:20:07.861880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:20:07.863022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:20:07.864070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 665/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Mon May 9 18:20:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 141 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:20:07.868856 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776ebff-9d43-4e14-8bf6-33f018734f14 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:20:07.961276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776ebff-9d43-4e14-8bf6-33f018734f14 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:20:07.993714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c561fe-3016-4ee8-a73e-e12042e10842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:20:07.994376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c561fe-3016-4ee8-a73e-e12042e10842 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:07.994660 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c561fe-3016-4ee8-a73e-e12042e10842 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:07.994938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c561fe-3016-4ee8-a73e-e12042e10842 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:07.995667 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c561fe-3016-4ee8-a73e-e12042e10842 None admin] RBAC: Adding query filter params (name=placement) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:20:07.995973 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c561fe-3016-4ee8-a73e-e12042e10842 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:20:07.997317 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c561fe-3016-4ee8-a73e-e12042e10842 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:20:08.002508 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 634/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Mon May 9 18:20:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:20:08.008442 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297d504-53ed-4ed5-89b2-43bba473dbda None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:20:08.107633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297d504-53ed-4ed5-89b2-43bba473dbda None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:20:08.141794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90199f-2b86-4649-9f48-b39ae7e94d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:20:08.142370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90199f-2b86-4649-9f48-b39ae7e94d8a None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:08.142649 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90199f-2b86-4649-9f48-b39ae7e94d8a None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:08.142921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90199f-2b86-4649-9f48-b39ae7e94d8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:08.143720 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90199f-2b86-4649-9f48-b39ae7e94d8a None admin] RBAC: Adding query filter params (interface=public, service_id=06d534142125449b9065c3a5dbe3dcd8, region_id=RegionOne) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:20:08.144027 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90199f-2b86-4649-9f48-b39ae7e94d8a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:20:08.145359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90199f-2b86-4649-9f48-b39ae7e94d8a None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:20:08.150705 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 666/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Mon May 9 18:20:08 2022] GET /identity/v3/endpoints?service_id=06d534142125449b9065c3a5dbe3dcd8&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:20:08.156418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c561fe-3016-4ee8-a73e-e12042e10842 None admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:20:08.252389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c561fe-3016-4ee8-a73e-e12042e10842 None admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:20:08.289046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabb5ce-d7a3-4e82-bb06-f66358285663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:20:08.289738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabb5ce-d7a3-4e82-bb06-f66358285663 None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:08.290019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabb5ce-d7a3-4e82-bb06-f66358285663 None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:08.290294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabb5ce-d7a3-4e82-bb06-f66358285663 None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:08.290962 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabb5ce-d7a3-4e82-bb06-f66358285663 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:20:08.292340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabb5ce-d7a3-4e82-bb06-f66358285663 None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:20:08.298833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 635/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Mon May 9 18:20:08 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:20:10.471516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340273d9-4dc8-47ea-ae46-734bd47ac140 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:10.472113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340273d9-4dc8-47ea-ae46-734bd47ac140 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:10.472113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340273d9-4dc8-47ea-ae46-734bd47ac140 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:10.473393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 667/1302] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:20:10.479042 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dac3b5-a7d3-48c8-b3ba-22defc007c31 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:10.479473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dac3b5-a7d3-48c8-b3ba-22defc007c31 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:10.479731 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dac3b5-a7d3-48c8-b3ba-22defc007c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:10.496988 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-e5dac3b5-a7d3-48c8-b3ba-22defc007c31 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:20:10.498393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e5dac3b5-a7d3-48c8-b3ba-22defc007c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:20:10.559729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 636/1303] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:20:10 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 18:20:10.581713 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340273d9-4dc8-47ea-ae46-734bd47ac140 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:20:10.676146 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340273d9-4dc8-47ea-ae46-734bd47ac140 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:20:10.707351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e630e72-f5e6-4b03-8024-3dcc63811757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:20:10.708136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e630e72-f5e6-4b03-8024-3dcc63811757 None admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:10.708411 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e630e72-f5e6-4b03-8024-3dcc63811757 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:10.708681 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e630e72-f5e6-4b03-8024-3dcc63811757 None admin] PATH_INFO: `/v3/services/placement` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:10.709308 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e630e72-f5e6-4b03-8024-3dcc63811757 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:20:10.711383 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e630e72-f5e6-4b03-8024-3dcc63811757 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application [None req-7e630e72-f5e6-4b03-8024-3dcc63811757 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 18:20:10.723801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 18:20:10.724948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.get_or_create( May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application with Lock( May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self._enter() May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application return self.creator() May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application created_value = creator( May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: ERROR keystone.server.flask.application May 09 18:20:10.725856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 668/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Mon May 9 18:20:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 18:20:10.731869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dac3b5-a7d3-48c8-b3ba-22defc007c31 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:20:10.823262 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dac3b5-a7d3-48c8-b3ba-22defc007c31 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:20:10.855188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c301e4-dbd6-4dfc-ae84-253a530d554f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:20:10.855866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c301e4-dbd6-4dfc-ae84-253a530d554f None admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:10.856151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c301e4-dbd6-4dfc-ae84-253a530d554f None admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:10.856428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c301e4-dbd6-4dfc-ae84-253a530d554f None admin] PATH_INFO: `/v3/services` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:10.857121 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c301e4-dbd6-4dfc-ae84-253a530d554f None admin] RBAC: Adding query filter params (name=placement) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:20:10.857426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c301e4-dbd6-4dfc-ae84-253a530d554f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:20:10.858775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c301e4-dbd6-4dfc-ae84-253a530d554f None admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:20:10.863693 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 637/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Mon May 9 18:20:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:20:10.869208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e630e72-f5e6-4b03-8024-3dcc63811757 None admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:20:10.961033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e630e72-f5e6-4b03-8024-3dcc63811757 None admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:20:10.992512 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef643aeb-aa6b-4770-bfeb-868d80f30428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:20:10.993147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef643aeb-aa6b-4770-bfeb-868d80f30428 None admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:20:10.993421 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef643aeb-aa6b-4770-bfeb-868d80f30428 None admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:20:10.993687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef643aeb-aa6b-4770-bfeb-868d80f30428 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:20:10.994393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef643aeb-aa6b-4770-bfeb-868d80f30428 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:20:10.995752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef643aeb-aa6b-4770-bfeb-868d80f30428 None admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:20:11.012814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 669/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Mon May 9 18:20:10 2022] POST /identity/v3/endpoints => generated 331 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:21:14.255448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9873b17-3588-445e-bd8c-8b442d238c15 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:14.256367 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9873b17-3588-445e-bd8c-8b442d238c15 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:14.256801 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9873b17-3588-445e-bd8c-8b442d238c15 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:14.263608 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde851b-dbe3-4b0b-854f-9496877adb06 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:14.264417 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde851b-dbe3-4b0b-854f-9496877adb06 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:14.264850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde851b-dbe3-4b0b-854f-9496877adb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:14.268583 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 638/1307] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:21:14 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:14.337159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-7cde851b-dbe3-4b0b-854f-9496877adb06 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:14.338613 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7cde851b-dbe3-4b0b-854f-9496877adb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:14.503563 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 670/1308] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:21:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:21:14.552113 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562a956-8a98-4362-a4d0-af737c0974f5 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:14.552113 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562a956-8a98-4362-a4d0-af737c0974f5 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:14.552113 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562a956-8a98-4362-a4d0-af737c0974f5 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:14.552113 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 639/1309] 10.222.0.25 () {54 vars in 964 bytes} [Mon May 9 18:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:14.562432 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44759c6-ddff-422a-a05f-628d07339979 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:14.563316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44759c6-ddff-422a-a05f-628d07339979 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:14.564626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44759c6-ddff-422a-a05f-628d07339979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:14.674921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c44759c6-ddff-422a-a05f-628d07339979 None None] MFA Rules not processed for user `6b2b6318776b422d9701351ba6b4ee27`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:14.676987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c44759c6-ddff-422a-a05f-628d07339979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:14.759469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 671/1310] 10.222.0.25 () {58 vars in 1061 bytes} [Mon May 9 18:21:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:21:14.768179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562a956-8a98-4362-a4d0-af737c0974f5 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:21:14.770534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1562a956-8a98-4362-a4d0-af737c0974f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:14.888931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562a956-8a98-4362-a4d0-af737c0974f5 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:21:14.962812 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bac9d5-2f00-413a-a1b6-8bc82bdbebcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:21:14.963514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bac9d5-2f00-413a-a1b6-8bc82bdbebcf service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:14.963805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bac9d5-2f00-413a-a1b6-8bc82bdbebcf service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:14.964075 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bac9d5-2f00-413a-a1b6-8bc82bdbebcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:14.968056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-31bac9d5-2f00-413a-a1b6-8bc82bdbebcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:15.075447 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bac9d5-2f00-413a-a1b6-8bc82bdbebcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:21:15.076792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bac9d5-2f00-413a-a1b6-8bc82bdbebcf service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:21:15.095118 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 640/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:21:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:21:17.629797 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de9181-3478-470b-8b2b-0771daf596f8 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:17.630396 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de9181-3478-470b-8b2b-0771daf596f8 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:17.630396 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de9181-3478-470b-8b2b-0771daf596f8 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:17.631926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 672/1312] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:21:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:17.637186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e7c9e-e293-4160-8fec-bb5ac584ba35 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:17.638089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e7c9e-e293-4160-8fec-bb5ac584ba35 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:17.638471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e7c9e-e293-4160-8fec-bb5ac584ba35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:17.728109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-458e7c9e-e293-4160-8fec-bb5ac584ba35 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:17.729732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-458e7c9e-e293-4160-8fec-bb5ac584ba35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:17.851227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 641/1313] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:21:17.883570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de9181-3478-470b-8b2b-0771daf596f8 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:21:17.885168 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e0de9181-3478-470b-8b2b-0771daf596f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:17.992991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de9181-3478-470b-8b2b-0771daf596f8 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:21:18.067273 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955554b4-a40b-47ee-a41b-c702581b984d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:21:18.067930 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955554b4-a40b-47ee-a41b-c702581b984d service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:18.068214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955554b4-a40b-47ee-a41b-c702581b984d service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:18.068492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955554b4-a40b-47ee-a41b-c702581b984d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:18.070122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-955554b4-a40b-47ee-a41b-c702581b984d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:18.176218 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955554b4-a40b-47ee-a41b-c702581b984d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:21:18.177577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955554b4-a40b-47ee-a41b-c702581b984d service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:21:18.197108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 673/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:21:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:21:20.754024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e40aa-4f48-47c3-83e6-9a69c136335b None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:20.754573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e40aa-4f48-47c3-83e6-9a69c136335b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:20.754573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e40aa-4f48-47c3-83e6-9a69c136335b None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:20.755958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 642/1315] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:20.761025 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb6611-336c-4e19-958f-7219066a8ec0 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:20.761451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb6611-336c-4e19-958f-7219066a8ec0 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:20.761568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb6611-336c-4e19-958f-7219066a8ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:20.822985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-3bcb6611-336c-4e19-958f-7219066a8ec0 None None] MFA Rules not processed for user `416a6d6800844d94a22ac939f6f6ba2a`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:20.824245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3bcb6611-336c-4e19-958f-7219066a8ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:20.925712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 674/1316] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:21:20 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:21:20.980141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93496ee3-6b86-4e48-8c38-7e82610609f1 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:20.980459 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93496ee3-6b86-4e48-8c38-7e82610609f1 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:20.980738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93496ee3-6b86-4e48-8c38-7e82610609f1 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:20.982098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 643/1317] 10.222.0.25 () {54 vars in 964 bytes} [Mon May 9 18:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:20.988560 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7d3c6-34be-4c8b-8076-b18be6896ef3 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:20.988844 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7d3c6-34be-4c8b-8076-b18be6896ef3 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:20.989122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7d3c6-34be-4c8b-8076-b18be6896ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:21.059688 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-89d7d3c6-34be-4c8b-8076-b18be6896ef3 None None] MFA Rules not processed for user `6b2b6318776b422d9701351ba6b4ee27`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:21.060800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-89d7d3c6-34be-4c8b-8076-b18be6896ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:21.130948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 675/1318] 10.222.0.25 () {58 vars in 1061 bytes} [Mon May 9 18:21:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:21:21.139802 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93496ee3-6b86-4e48-8c38-7e82610609f1 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:21:21.141300 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-93496ee3-6b86-4e48-8c38-7e82610609f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:21.238701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93496ee3-6b86-4e48-8c38-7e82610609f1 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:21:21.306700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6256c-ec9d-4ffb-9c92-3b1efb25225f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:21:21.307366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6256c-ec9d-4ffb-9c92-3b1efb25225f service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:21.307648 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6256c-ec9d-4ffb-9c92-3b1efb25225f service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:21.307921 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6256c-ec9d-4ffb-9c92-3b1efb25225f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:21.309519 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-85a6256c-ec9d-4ffb-9c92-3b1efb25225f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:21.411131 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a6256c-ec9d-4ffb-9c92-3b1efb25225f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:21:21.412503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a6256c-ec9d-4ffb-9c92-3b1efb25225f service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:21:21.429892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 644/1319] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:21:21 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:21:24.591965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956b315-ad44-4efc-a0ed-8e5d34f98d59 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:24.592558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956b315-ad44-4efc-a0ed-8e5d34f98d59 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:24.592558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956b315-ad44-4efc-a0ed-8e5d34f98d59 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:24.593871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 676/1320] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:24.599697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6129c8-f5b0-4a73-90c6-c4dbe159fe28 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:24.600154 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6129c8-f5b0-4a73-90c6-c4dbe159fe28 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:24.600353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6129c8-f5b0-4a73-90c6-c4dbe159fe28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:24.663315 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-8a6129c8-f5b0-4a73-90c6-c4dbe159fe28 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:24.664499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-8a6129c8-f5b0-4a73-90c6-c4dbe159fe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:24.756343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 645/1321] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:21:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:21:24.795030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956b315-ad44-4efc-a0ed-8e5d34f98d59 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:21:24.796560 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-0956b315-ad44-4efc-a0ed-8e5d34f98d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:24.893693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956b315-ad44-4efc-a0ed-8e5d34f98d59 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:21:24.963728 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac1244-fcc0-4f07-b539-3ee2c69e6ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:21:24.964373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac1244-fcc0-4f07-b539-3ee2c69e6ffe service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:24.964654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac1244-fcc0-4f07-b539-3ee2c69e6ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:24.964928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac1244-fcc0-4f07-b539-3ee2c69e6ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:24.966583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-97ac1244-fcc0-4f07-b539-3ee2c69e6ffe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:25.070341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac1244-fcc0-4f07-b539-3ee2c69e6ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:21:25.071800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac1244-fcc0-4f07-b539-3ee2c69e6ffe service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:21:25.089274 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 677/1322] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:21:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:21:27.978323 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d328b-9a9a-4f10-b089-6ecbc8bc043b None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:27.978792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d328b-9a9a-4f10-b089-6ecbc8bc043b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:27.978897 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d328b-9a9a-4f10-b089-6ecbc8bc043b None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:27.980254 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 646/1323] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:27.985360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bb80f-1790-43dd-a853-05deb01695f0 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:27.985790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bb80f-1790-43dd-a853-05deb01695f0 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:27.985915 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bb80f-1790-43dd-a853-05deb01695f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:28.051931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-935bb80f-1790-43dd-a853-05deb01695f0 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:28.053104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-935bb80f-1790-43dd-a853-05deb01695f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:28.143657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 678/1324] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:21:28.177810 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d328b-9a9a-4f10-b089-6ecbc8bc043b None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:21:28.179335 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-5a8d328b-9a9a-4f10-b089-6ecbc8bc043b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:28.278682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d328b-9a9a-4f10-b089-6ecbc8bc043b None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:21:28.348202 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de9cbd-cea3-4ee0-bfa1-fcb6d32adb8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:21:28.348859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de9cbd-cea3-4ee0-bfa1-fcb6d32adb8e service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:28.349138 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de9cbd-cea3-4ee0-bfa1-fcb6d32adb8e service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:28.349412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de9cbd-cea3-4ee0-bfa1-fcb6d32adb8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:28.351036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e9de9cbd-cea3-4ee0-bfa1-fcb6d32adb8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:28.453718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de9cbd-cea3-4ee0-bfa1-fcb6d32adb8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:21:28.455136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de9cbd-cea3-4ee0-bfa1-fcb6d32adb8e service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:21:28.472883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 647/1325] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:21:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:21:30.745715 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19575a2-6afc-4774-94c9-14ea4a1f95dc None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:30.746299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19575a2-6afc-4774-94c9-14ea4a1f95dc None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:30.746299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19575a2-6afc-4774-94c9-14ea4a1f95dc None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:30.747842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 679/1326] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:30.753180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d187446-1a7c-4649-9b91-6117d6fde584 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:30.753595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d187446-1a7c-4649-9b91-6117d6fde584 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:30.753860 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d187446-1a7c-4649-9b91-6117d6fde584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:30.816499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-8d187446-1a7c-4649-9b91-6117d6fde584 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:30.817746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-8d187446-1a7c-4649-9b91-6117d6fde584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:30.912834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 648/1327] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:21:30.947964 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19575a2-6afc-4774-94c9-14ea4a1f95dc None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:21:30.949467 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b19575a2-6afc-4774-94c9-14ea4a1f95dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:31.056776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19575a2-6afc-4774-94c9-14ea4a1f95dc None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:21:31.125996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d94e5-34d8-4cae-b90b-8498f3b0c242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:21:31.126643 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d94e5-34d8-4cae-b90b-8498f3b0c242 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:31.126923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d94e5-34d8-4cae-b90b-8498f3b0c242 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:31.127735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d94e5-34d8-4cae-b90b-8498f3b0c242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:31.129404 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-408d94e5-34d8-4cae-b90b-8498f3b0c242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:31.231996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d94e5-34d8-4cae-b90b-8498f3b0c242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:21:31.233416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d94e5-34d8-4cae-b90b-8498f3b0c242 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:21:31.250843 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 680/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:21:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:21:33.510541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384f559-5789-4247-b6d0-c835041b15fb None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:33.511104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384f559-5789-4247-b6d0-c835041b15fb None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:33.511104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384f559-5789-4247-b6d0-c835041b15fb None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:33.512503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 649/1329] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:33.518020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f93ba-9937-4223-b056-ef65f77f5543 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:33.518455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f93ba-9937-4223-b056-ef65f77f5543 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:33.518577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f93ba-9937-4223-b056-ef65f77f5543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:33.580497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-6e9f93ba-9937-4223-b056-ef65f77f5543 None None] MFA Rules not processed for user `416a6d6800844d94a22ac939f6f6ba2a`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:33.581714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-6e9f93ba-9937-4223-b056-ef65f77f5543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:33.675863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 681/1330] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:21:33 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:21:33.711750 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384f559-5789-4247-b6d0-c835041b15fb None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:21:33.713302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7384f559-5789-4247-b6d0-c835041b15fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:33.816215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384f559-5789-4247-b6d0-c835041b15fb None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:21:33.886666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f275349-427f-4395-9eeb-6e66822b3cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:21:33.887346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f275349-427f-4395-9eeb-6e66822b3cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:33.887626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f275349-427f-4395-9eeb-6e66822b3cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:33.887895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f275349-427f-4395-9eeb-6e66822b3cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:33.889484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1f275349-427f-4395-9eeb-6e66822b3cd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:33.994836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f275349-427f-4395-9eeb-6e66822b3cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:21:33.996553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f275349-427f-4395-9eeb-6e66822b3cd0 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:21:34.013819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 650/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:21:33 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:21:36.437444 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b1a17-e54f-4bc8-9fa4-698239e76135 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:36.437898 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b1a17-e54f-4bc8-9fa4-698239e76135 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:36.437975 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b1a17-e54f-4bc8-9fa4-698239e76135 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:36.439404 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 682/1332] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:36.445485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217503c1-7907-4be6-85dd-8fadf41d3557 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:36.445485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217503c1-7907-4be6-85dd-8fadf41d3557 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:36.445962 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217503c1-7907-4be6-85dd-8fadf41d3557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:36.514847 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-217503c1-7907-4be6-85dd-8fadf41d3557 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:36.516076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-217503c1-7907-4be6-85dd-8fadf41d3557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:36.618062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 651/1333] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:21:36.653094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b1a17-e54f-4bc8-9fa4-698239e76135 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:21:36.654647 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-337b1a17-e54f-4bc8-9fa4-698239e76135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:36.752864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b1a17-e54f-4bc8-9fa4-698239e76135 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:21:36.822301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c71b30-36bf-4b5a-b92f-ebed2f472359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:21:36.822948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c71b30-36bf-4b5a-b92f-ebed2f472359 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:36.823259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c71b30-36bf-4b5a-b92f-ebed2f472359 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:36.823532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c71b30-36bf-4b5a-b92f-ebed2f472359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:36.825153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a2c71b30-36bf-4b5a-b92f-ebed2f472359 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:36.929130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c71b30-36bf-4b5a-b92f-ebed2f472359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:21:36.930478 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c71b30-36bf-4b5a-b92f-ebed2f472359 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:21:36.947796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 683/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:21:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:21:39.224409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adb84c-ac1b-48cb-892c-62e7d8df2e10 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:39.224930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adb84c-ac1b-48cb-892c-62e7d8df2e10 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:39.224930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adb84c-ac1b-48cb-892c-62e7d8df2e10 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:39.226300 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 652/1335] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:39.231540 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a189fb-fdec-4789-a7f0-c50b56032c10 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:39.231926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a189fb-fdec-4789-a7f0-c50b56032c10 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:39.232136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a189fb-fdec-4789-a7f0-c50b56032c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:39.296620 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c7a189fb-fdec-4789-a7f0-c50b56032c10 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:39.297818 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c7a189fb-fdec-4789-a7f0-c50b56032c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:39.388406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 684/1336] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:21:39.430325 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adb84c-ac1b-48cb-892c-62e7d8df2e10 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:21:39.431916 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-81adb84c-ac1b-48cb-892c-62e7d8df2e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:39.531786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adb84c-ac1b-48cb-892c-62e7d8df2e10 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:21:39.600722 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a840cd-0a45-48ff-bc68-ea260ffdf633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:21:39.601388 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a840cd-0a45-48ff-bc68-ea260ffdf633 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:39.601668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a840cd-0a45-48ff-bc68-ea260ffdf633 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:39.601940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a840cd-0a45-48ff-bc68-ea260ffdf633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:39.603573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-85a840cd-0a45-48ff-bc68-ea260ffdf633 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:39.705943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a840cd-0a45-48ff-bc68-ea260ffdf633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:21:39.707370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a840cd-0a45-48ff-bc68-ea260ffdf633 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:21:39.724163 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 653/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:21:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:21:42.254198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be4e6c-b435-4bdd-956f-5ff84fa16856 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:42.254696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be4e6c-b435-4bdd-956f-5ff84fa16856 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:42.254806 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be4e6c-b435-4bdd-956f-5ff84fa16856 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:42.256160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 685/1338] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:42.261347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018dd29-cc21-4b58-9467-535070e577dc None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:42.261763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018dd29-cc21-4b58-9467-535070e577dc None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:42.261907 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018dd29-cc21-4b58-9467-535070e577dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:42.323346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-d018dd29-cc21-4b58-9467-535070e577dc None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:42.324518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-d018dd29-cc21-4b58-9467-535070e577dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:42.415492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 654/1339] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:21:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:21:42.453955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be4e6c-b435-4bdd-956f-5ff84fa16856 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:21:42.455517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-69be4e6c-b435-4bdd-956f-5ff84fa16856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:42.553062 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be4e6c-b435-4bdd-956f-5ff84fa16856 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:21:42.623427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36bf148-5622-4202-845b-1248e6924229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:21:42.624068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36bf148-5622-4202-845b-1248e6924229 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:42.624349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36bf148-5622-4202-845b-1248e6924229 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:42.624623 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36bf148-5622-4202-845b-1248e6924229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:42.626233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a36bf148-5622-4202-845b-1248e6924229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:42.728323 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36bf148-5622-4202-845b-1248e6924229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:21:42.729727 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36bf148-5622-4202-845b-1248e6924229 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:21:42.747473 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 686/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:21:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:21:46.916628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9408f3-1f0e-4a5d-bdba-34ba20165020 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:46.917134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9408f3-1f0e-4a5d-bdba-34ba20165020 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:46.917134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9408f3-1f0e-4a5d-bdba-34ba20165020 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:46.918804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 655/1341] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:46.924305 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a637c1c-d69a-4435-85fa-0a6d4beaf0b0 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:46.924758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a637c1c-d69a-4435-85fa-0a6d4beaf0b0 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:46.924975 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a637c1c-d69a-4435-85fa-0a6d4beaf0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:46.998956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4a637c1c-d69a-4435-85fa-0a6d4beaf0b0 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:47.000165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4a637c1c-d69a-4435-85fa-0a6d4beaf0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:47.116025 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 687/1342] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:21:47.156672 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9408f3-1f0e-4a5d-bdba-34ba20165020 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:21:47.159120 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-dd9408f3-1f0e-4a5d-bdba-34ba20165020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:47.253255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9408f3-1f0e-4a5d-bdba-34ba20165020 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:21:47.256800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073540b6-bb9f-412a-92d7-f6f2ff8da10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:21:47.257446 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073540b6-bb9f-412a-92d7-f6f2ff8da10b service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:47.257725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073540b6-bb9f-412a-92d7-f6f2ff8da10b service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:47.257997 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073540b6-bb9f-412a-92d7-f6f2ff8da10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:47.266880 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073540b6-bb9f-412a-92d7-f6f2ff8da10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:21:47.268341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073540b6-bb9f-412a-92d7-f6f2ff8da10b service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:21:47.277893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 656/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:21:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:21:50.100697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cebe76-fa24-460e-b037-d89e7c15050c None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:50.100697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cebe76-fa24-460e-b037-d89e7c15050c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:50.101352 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cebe76-fa24-460e-b037-d89e7c15050c None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:50.102719 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 688/1344] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:21:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:21:50.109406 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97642e07-7a5d-44e1-a14e-9353f3a3e0ef None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:50.109867 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97642e07-7a5d-44e1-a14e-9353f3a3e0ef None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:50.109981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97642e07-7a5d-44e1-a14e-9353f3a3e0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:50.138252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-97642e07-7a5d-44e1-a14e-9353f3a3e0ef None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:50.139472 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-97642e07-7a5d-44e1-a14e-9353f3a3e0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:50.212348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 657/1345] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:21:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:21:50.249864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cebe76-fa24-460e-b037-d89e7c15050c None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:21:50.260266 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cebe76-fa24-460e-b037-d89e7c15050c None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:21:50.263698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c29652-86a4-4097-8bec-24b77a9009e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:21:50.264327 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c29652-86a4-4097-8bec-24b77a9009e7 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:50.264602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c29652-86a4-4097-8bec-24b77a9009e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:50.264876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c29652-86a4-4097-8bec-24b77a9009e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:50.274236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c29652-86a4-4097-8bec-24b77a9009e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:21:50.275678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c29652-86a4-4097-8bec-24b77a9009e7 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:21:50.285541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 689/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:21:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:21:54.463985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbf29e-a0fa-4539-a0ba-4e6ac4207b49 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:54.464556 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbf29e-a0fa-4539-a0ba-4e6ac4207b49 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:54.464556 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbf29e-a0fa-4539-a0ba-4e6ac4207b49 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:54.465904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 658/1347] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:54.471549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394155-754e-452a-8c90-9e878f3d7c76 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:54.471995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394155-754e-452a-8c90-9e878f3d7c76 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:54.472136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394155-754e-452a-8c90-9e878f3d7c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:54.489770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-f4394155-754e-452a-8c90-9e878f3d7c76 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:21:54.490945 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f4394155-754e-452a-8c90-9e878f3d7c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:21:54.498967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 690/1348] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:21:54.536302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbf29e-a0fa-4539-a0ba-4e6ac4207b49 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:21:54.546945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbf29e-a0fa-4539-a0ba-4e6ac4207b49 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:21:54.551229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd37f95-ac35-46e7-89a9-444edb20624c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:21:54.551878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd37f95-ac35-46e7-89a9-444edb20624c service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:54.552228 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd37f95-ac35-46e7-89a9-444edb20624c service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:54.552585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd37f95-ac35-46e7-89a9-444edb20624c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:54.561365 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd37f95-ac35-46e7-89a9-444edb20624c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:21:54.562840 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd37f95-ac35-46e7-89a9-444edb20624c service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:21:54.572760 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 659/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:21:59.928131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c0bad-6cb2-45f1-a859-52e3a1155dcd None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:59.928891 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c0bad-6cb2-45f1-a859-52e3a1155dcd None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:59.929319 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c0bad-6cb2-45f1-a859-52e3a1155dcd None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:21:59.931078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 691/1350] 10.222.0.25 () {54 vars in 921 bytes} [Mon May 9 18:21:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:21:59.936197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff0a61-6e26-45a4-be9f-6581f9c5d401 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:21:59.936943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff0a61-6e26-45a4-be9f-6581f9c5d401 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:21:59.937404 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff0a61-6e26-45a4-be9f-6581f9c5d401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:00.012130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-0eff0a61-6e26-45a4-be9f-6581f9c5d401 None None] MFA Rules not processed for user `f747dcb955fa499a972c46587d3db294`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:00.013778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-0eff0a61-6e26-45a4-be9f-6581f9c5d401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:00.090779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 660/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Mon May 9 18:21:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:04.308907 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7030f-6bdc-4f6b-9fa6-ee4550b217f0 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:04.310015 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7030f-6bdc-4f6b-9fa6-ee4550b217f0 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:04.310460 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7030f-6bdc-4f6b-9fa6-ee4550b217f0 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:04.311938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 692/1352] 10.222.0.25 () {54 vars in 921 bytes} [Mon May 9 18:22:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:04.317522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7799d-e861-4192-aa5e-d26437a573cd None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:04.318038 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7799d-e861-4192-aa5e-d26437a573cd None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:04.318290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7799d-e861-4192-aa5e-d26437a573cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:04.336837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-3bb7799d-e861-4192-aa5e-d26437a573cd None None] MFA Rules not processed for user `f747dcb955fa499a972c46587d3db294`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:04.338131 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-3bb7799d-e861-4192-aa5e-d26437a573cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:04.347316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 661/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Mon May 9 18:22:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:06.047819 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb52f2-3f2e-4749-8a8e-637c6235a827 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:06.048347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb52f2-3f2e-4749-8a8e-637c6235a827 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:06.048347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb52f2-3f2e-4749-8a8e-637c6235a827 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:06.049766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 693/1354] 10.222.0.25 () {54 vars in 921 bytes} [Mon May 9 18:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:06.054499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17310736-ca23-4c84-b994-8b27f12ef7a5 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:06.054948 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17310736-ca23-4c84-b994-8b27f12ef7a5 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:06.055077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17310736-ca23-4c84-b994-8b27f12ef7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:06.072637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-17310736-ca23-4c84-b994-8b27f12ef7a5 None None] MFA Rules not processed for user `f747dcb955fa499a972c46587d3db294`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:06.073929 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-17310736-ca23-4c84-b994-8b27f12ef7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:06.083736 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 662/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Mon May 9 18:22:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:22:09.899585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d3d01-564d-4fae-a379-61142ebd7954 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:09.900128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d3d01-564d-4fae-a379-61142ebd7954 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:09.900128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d3d01-564d-4fae-a379-61142ebd7954 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:09.901396 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 694/1356] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:09.906837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30a425-4e2d-4f05-898e-27405472bfbb None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:09.907324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30a425-4e2d-4f05-898e-27405472bfbb None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:09.907525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30a425-4e2d-4f05-898e-27405472bfbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:09.925378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ed30a425-4e2d-4f05-898e-27405472bfbb None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:09.926632 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ed30a425-4e2d-4f05-898e-27405472bfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:09.934513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 663/1357] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:22:09.982155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841e1e0-c3d8-4621-83da-81ad6a7e39c5 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:09.982444 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841e1e0-c3d8-4621-83da-81ad6a7e39c5 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:09.982721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841e1e0-c3d8-4621-83da-81ad6a7e39c5 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:09.984097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 695/1358] 10.222.0.25 () {54 vars in 961 bytes} [Mon May 9 18:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:09.990151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43899116-ef00-48e5-848c-ba9a0edbbad8 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:09.990444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43899116-ef00-48e5-848c-ba9a0edbbad8 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:09.990719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43899116-ef00-48e5-848c-ba9a0edbbad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:10.042030 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-43899116-ef00-48e5-848c-ba9a0edbbad8 None None] MFA Rules not processed for user `f769ee9f31354e5296645fb1e3bd84b6`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:10.043345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-43899116-ef00-48e5-848c-ba9a0edbbad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:10.130409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 664/1359] 10.222.0.25 () {58 vars in 1058 bytes} [Mon May 9 18:22:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:10.142162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841e1e0-c3d8-4621-83da-81ad6a7e39c5 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:10.156563 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841e1e0-c3d8-4621-83da-81ad6a7e39c5 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:10.162508 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a62efc-f14b-4e6d-a513-4702ba545686 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:10.163137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a62efc-f14b-4e6d-a513-4702ba545686 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:10.163446 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a62efc-f14b-4e6d-a513-4702ba545686 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:10.163720 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a62efc-f14b-4e6d-a513-4702ba545686 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:10.171567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a62efc-f14b-4e6d-a513-4702ba545686 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:10.173134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a62efc-f14b-4e6d-a513-4702ba545686 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:10.183148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 696/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:22:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:22:12.632348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba81b8-36a0-492b-b72a-4e25463308ab None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:12.632873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba81b8-36a0-492b-b72a-4e25463308ab None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:12.632954 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba81b8-36a0-492b-b72a-4e25463308ab None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:12.634439 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 665/1361] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:12.639602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a425f-e833-45cf-a34e-42b0ff7018a8 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:12.640061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a425f-e833-45cf-a34e-42b0ff7018a8 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:12.640343 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a425f-e833-45cf-a34e-42b0ff7018a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:12.658515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-a43a425f-e833-45cf-a34e-42b0ff7018a8 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:12.659726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a43a425f-e833-45cf-a34e-42b0ff7018a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:12.667792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 697/1362] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:12.711703 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba81b8-36a0-492b-b72a-4e25463308ab None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:12.726598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba81b8-36a0-492b-b72a-4e25463308ab None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:12.733991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ee20f-73e5-4e0b-ae22-2b4df4f2b95c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:12.734857 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ee20f-73e5-4e0b-ae22-2b4df4f2b95c service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:12.735367 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ee20f-73e5-4e0b-ae22-2b4df4f2b95c service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:12.735713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ee20f-73e5-4e0b-ae22-2b4df4f2b95c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:12.746411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ee20f-73e5-4e0b-ae22-2b4df4f2b95c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:12.748061 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ee20f-73e5-4e0b-ae22-2b4df4f2b95c service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:12.758728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 666/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:22:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:22:15.314888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62f143-d783-46ec-a3f6-f74450e9ecec None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:15.315387 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62f143-d783-46ec-a3f6-f74450e9ecec None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:15.315471 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62f143-d783-46ec-a3f6-f74450e9ecec None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:15.316841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 698/1364] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:15.322380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a078e16-4f9a-4e51-8f54-fdbf34ec7771 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:15.322808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a078e16-4f9a-4e51-8f54-fdbf34ec7771 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:15.322939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a078e16-4f9a-4e51-8f54-fdbf34ec7771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:15.343351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-1a078e16-4f9a-4e51-8f54-fdbf34ec7771 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:15.344553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1a078e16-4f9a-4e51-8f54-fdbf34ec7771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:15.354475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 667/1365] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:15.406912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a2fc9-8814-4516-b16c-8449375d5cf9 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:15.407627 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a2fc9-8814-4516-b16c-8449375d5cf9 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:15.408146 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a2fc9-8814-4516-b16c-8449375d5cf9 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:15.410554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 699/1366] 10.222.0.25 () {54 vars in 961 bytes} [Mon May 9 18:22:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:15.416210 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73880b8-4ec6-4913-b1e9-bdb1c6264435 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:15.416450 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73880b8-4ec6-4913-b1e9-bdb1c6264435 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:15.416713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73880b8-4ec6-4913-b1e9-bdb1c6264435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:15.438517 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-b73880b8-4ec6-4913-b1e9-bdb1c6264435 None None] MFA Rules not processed for user `f769ee9f31354e5296645fb1e3bd84b6`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:15.440617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b73880b8-4ec6-4913-b1e9-bdb1c6264435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:15.449852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 668/1367] 10.222.0.25 () {58 vars in 1058 bytes} [Mon May 9 18:22:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:15.458203 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a2fc9-8814-4516-b16c-8449375d5cf9 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:15.472004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a2fc9-8814-4516-b16c-8449375d5cf9 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:15.477383 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91bd23-9083-462f-b74f-a91bd3829392 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:15.478007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91bd23-9083-462f-b74f-a91bd3829392 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:15.478284 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91bd23-9083-462f-b74f-a91bd3829392 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:15.478556 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91bd23-9083-462f-b74f-a91bd3829392 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:15.486378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91bd23-9083-462f-b74f-a91bd3829392 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:15.488014 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91bd23-9083-462f-b74f-a91bd3829392 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:15.497659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 700/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:22:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:22:17.786863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008413da-5260-419b-a830-3cf26d3b90c6 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:17.787490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008413da-5260-419b-a830-3cf26d3b90c6 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:17.787490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008413da-5260-419b-a830-3cf26d3b90c6 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:17.789072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 669/1369] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:17.794259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bdb36-db50-416c-9334-305719c684df None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:17.795100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bdb36-db50-416c-9334-305719c684df None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:17.795429 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bdb36-db50-416c-9334-305719c684df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:17.816195 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4e8bdb36-db50-416c-9334-305719c684df None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:17.817380 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4e8bdb36-db50-416c-9334-305719c684df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:17.825474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 701/1370] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:17.869822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008413da-5260-419b-a830-3cf26d3b90c6 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:17.884314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008413da-5260-419b-a830-3cf26d3b90c6 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:17.890000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02f7ad-a625-482c-a018-5561db67b77b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:17.890645 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02f7ad-a625-482c-a018-5561db67b77b service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:17.890926 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02f7ad-a625-482c-a018-5561db67b77b service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:17.891237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02f7ad-a625-482c-a018-5561db67b77b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:17.899301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c02f7ad-a625-482c-a018-5561db67b77b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:17.900878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c02f7ad-a625-482c-a018-5561db67b77b service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:17.912203 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 670/1371] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:22:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:22:18.112682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa17d0-7cb0-4e11-a881-bdac13ecb469 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:18.112887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa17d0-7cb0-4e11-a881-bdac13ecb469 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:18.113119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa17d0-7cb0-4e11-a881-bdac13ecb469 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:18.114864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 702/1372] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:22:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:18.121664 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d5a3c-2bc1-4636-adb5-196b87d92cfd None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:18.122048 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d5a3c-2bc1-4636-adb5-196b87d92cfd None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:18.122431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d5a3c-2bc1-4636-adb5-196b87d92cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:18.148611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ae5d5a3c-2bc1-4636-adb5-196b87d92cfd None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:18.149878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ae5d5a3c-2bc1-4636-adb5-196b87d92cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:18.159072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 671/1373] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:18.467446 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38446fd4-1782-46ea-b914-4853115f840a None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:18.467732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38446fd4-1782-46ea-b914-4853115f840a None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:18.468008 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38446fd4-1782-46ea-b914-4853115f840a None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:18.469614 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 703/1374] 10.222.0.25 () {54 vars in 963 bytes} [Mon May 9 18:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:18.475193 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258a0ba-a34d-412d-9ffc-dbab6573ea6c None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:18.475501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258a0ba-a34d-412d-9ffc-dbab6573ea6c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:18.475779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258a0ba-a34d-412d-9ffc-dbab6573ea6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:18.524207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f258a0ba-a34d-412d-9ffc-dbab6573ea6c None None] MFA Rules not processed for user `4bda352a10f241ca96d5308a55238d57`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:18.525408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f258a0ba-a34d-412d-9ffc-dbab6573ea6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:18.587272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 672/1375] 10.222.0.25 () {58 vars in 1060 bytes} [Mon May 9 18:22:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:18.594911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38446fd4-1782-46ea-b914-4853115f840a None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:18.605744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38446fd4-1782-46ea-b914-4853115f840a None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:18.609232 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77790fba-3a92-491c-861e-125f365ebae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:18.609865 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77790fba-3a92-491c-861e-125f365ebae7 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:18.610359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77790fba-3a92-491c-861e-125f365ebae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:18.610637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77790fba-3a92-491c-861e-125f365ebae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:18.619615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77790fba-3a92-491c-861e-125f365ebae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:18.621043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77790fba-3a92-491c-861e-125f365ebae7 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:18.631694 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 704/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:22:20.183739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9489257-426a-41cb-886e-e562a78372cf None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:20.184289 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9489257-426a-41cb-886e-e562a78372cf None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:20.184289 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9489257-426a-41cb-886e-e562a78372cf None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:20.185653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 673/1377] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:20.192284 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02424038-3889-491f-b95b-ff26215c2eef None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:20.192704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02424038-3889-491f-b95b-ff26215c2eef None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:20.192704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02424038-3889-491f-b95b-ff26215c2eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:20.211402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-02424038-3889-491f-b95b-ff26215c2eef None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:20.212507 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-02424038-3889-491f-b95b-ff26215c2eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:20.220572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 705/1378] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:20.265592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9489257-426a-41cb-886e-e562a78372cf None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:20.280151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9489257-426a-41cb-886e-e562a78372cf None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:20.285741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87176f-cc8c-4d4d-87c8-5602c3c836e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:20.286416 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87176f-cc8c-4d4d-87c8-5602c3c836e1 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:20.286700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87176f-cc8c-4d4d-87c8-5602c3c836e1 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:20.286977 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87176f-cc8c-4d4d-87c8-5602c3c836e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:20.295861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87176f-cc8c-4d4d-87c8-5602c3c836e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:20.297473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87176f-cc8c-4d4d-87c8-5602c3c836e1 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:20.308258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 674/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:22:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:22:22.556402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcc8b3-9d21-43d9-bfd7-6f0867265b73 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:22.557272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcc8b3-9d21-43d9-bfd7-6f0867265b73 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:22.557546 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcc8b3-9d21-43d9-bfd7-6f0867265b73 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:22.559071 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 706/1380] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:22.564538 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06626ac3-609a-4a79-b6e4-4d1d9e51a6e4 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:22.564977 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06626ac3-609a-4a79-b6e4-4d1d9e51a6e4 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:22.565223 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06626ac3-609a-4a79-b6e4-4d1d9e51a6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:22.585228 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-06626ac3-609a-4a79-b6e4-4d1d9e51a6e4 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:22.586504 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-06626ac3-609a-4a79-b6e4-4d1d9e51a6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:22.594477 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 675/1381] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:22.638250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcc8b3-9d21-43d9-bfd7-6f0867265b73 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:22.653780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcc8b3-9d21-43d9-bfd7-6f0867265b73 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:22.659991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc4ea7-536d-418d-abd1-7e36f9043fb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:22.660630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4ea7-536d-418d-abd1-7e36f9043fb2 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:22.660907 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4ea7-536d-418d-abd1-7e36f9043fb2 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:22.661176 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4ea7-536d-418d-abd1-7e36f9043fb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:22.669596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc4ea7-536d-418d-abd1-7e36f9043fb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:22.671145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc4ea7-536d-418d-abd1-7e36f9043fb2 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:22.681215 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 707/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:22:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:22:24.931260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945daeea-d490-4ef7-8d4b-3ad5908ee517 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:24.931832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945daeea-d490-4ef7-8d4b-3ad5908ee517 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:24.931832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945daeea-d490-4ef7-8d4b-3ad5908ee517 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:24.933115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 676/1383] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:24.938266 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5349072-7399-4d9a-bd6d-db47f24f5c39 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:24.938679 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5349072-7399-4d9a-bd6d-db47f24f5c39 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:24.938811 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5349072-7399-4d9a-bd6d-db47f24f5c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:24.957101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-a5349072-7399-4d9a-bd6d-db47f24f5c39 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:24.958495 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a5349072-7399-4d9a-bd6d-db47f24f5c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:24.975213 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 708/1384] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:22:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:25.008166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945daeea-d490-4ef7-8d4b-3ad5908ee517 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:25.023480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945daeea-d490-4ef7-8d4b-3ad5908ee517 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:25.028832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6f026-acbf-4a5b-9c99-c437be8f3494 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:25.029487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6f026-acbf-4a5b-9c99-c437be8f3494 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:25.029770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6f026-acbf-4a5b-9c99-c437be8f3494 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:25.030044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6f026-acbf-4a5b-9c99-c437be8f3494 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:25.039861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6f026-acbf-4a5b-9c99-c437be8f3494 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:25.041673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6f026-acbf-4a5b-9c99-c437be8f3494 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:25.053410 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 677/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:22:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:22:27.318061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298d75d-6a1c-4e45-8258-997cfef9e808 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:27.318660 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298d75d-6a1c-4e45-8258-997cfef9e808 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:27.318770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298d75d-6a1c-4e45-8258-997cfef9e808 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:27.320653 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 709/1386] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:27.326058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865528f-2c51-44e4-85ec-8e7ce6e45a0a None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:27.326607 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865528f-2c51-44e4-85ec-8e7ce6e45a0a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:27.326726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865528f-2c51-44e4-85ec-8e7ce6e45a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:27.344504 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-6865528f-2c51-44e4-85ec-8e7ce6e45a0a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:27.345728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-6865528f-2c51-44e4-85ec-8e7ce6e45a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:27.354571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 678/1387] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:27.398047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298d75d-6a1c-4e45-8258-997cfef9e808 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:27.413795 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298d75d-6a1c-4e45-8258-997cfef9e808 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:27.419327 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae10fd-a049-4c28-8270-b345047655e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:27.419965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae10fd-a049-4c28-8270-b345047655e4 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:27.420248 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae10fd-a049-4c28-8270-b345047655e4 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:27.420546 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae10fd-a049-4c28-8270-b345047655e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:27.429533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae10fd-a049-4c28-8270-b345047655e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:27.431120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae10fd-a049-4c28-8270-b345047655e4 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:27.440667 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 710/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:22:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:22:29.777182 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00236-f831-4a99-a175-a52023d8c4ed None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:29.777726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00236-f831-4a99-a175-a52023d8c4ed None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:29.777726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00236-f831-4a99-a175-a52023d8c4ed None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:29.779109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 679/1389] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:29.786354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0d428-64c2-4a20-a99e-cdcfcadde985 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:29.786771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0d428-64c2-4a20-a99e-cdcfcadde985 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:29.787067 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0d428-64c2-4a20-a99e-cdcfcadde985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:29.808810 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-03c0d428-64c2-4a20-a99e-cdcfcadde985 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:29.810361 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-03c0d428-64c2-4a20-a99e-cdcfcadde985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:29.818882 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 711/1390] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:22:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:29.866761 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba00236-f831-4a99-a175-a52023d8c4ed None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:29.881370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba00236-f831-4a99-a175-a52023d8c4ed None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:29.891454 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372551f-6d4b-4ccf-8a47-d92b19339b2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:29.892093 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372551f-6d4b-4ccf-8a47-d92b19339b2c service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:29.892371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372551f-6d4b-4ccf-8a47-d92b19339b2c service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:29.892647 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372551f-6d4b-4ccf-8a47-d92b19339b2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:29.901321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372551f-6d4b-4ccf-8a47-d92b19339b2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:29.902923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372551f-6d4b-4ccf-8a47-d92b19339b2c service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:29.913618 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 680/1391] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:22:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:22:32.692676 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18f8af-dd4b-4447-a09a-dc55aa09f6b4 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:32.693288 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18f8af-dd4b-4447-a09a-dc55aa09f6b4 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:32.693288 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18f8af-dd4b-4447-a09a-dc55aa09f6b4 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:32.694557 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 712/1392] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:32.699762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c38a39-4a65-4cd7-ac14-bf889334ecae None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:32.700197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c38a39-4a65-4cd7-ac14-bf889334ecae None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:32.700298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c38a39-4a65-4cd7-ac14-bf889334ecae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:32.717364 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-02c38a39-4a65-4cd7-ac14-bf889334ecae None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:32.718527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-02c38a39-4a65-4cd7-ac14-bf889334ecae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:32.726649 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 681/1393] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:22:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:32.769808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18f8af-dd4b-4447-a09a-dc55aa09f6b4 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:32.784007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18f8af-dd4b-4447-a09a-dc55aa09f6b4 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:32.789303 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df599145-25e3-474e-82f5-81dcfef4a388 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:32.789930 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df599145-25e3-474e-82f5-81dcfef4a388 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:32.790206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df599145-25e3-474e-82f5-81dcfef4a388 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:32.790476 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df599145-25e3-474e-82f5-81dcfef4a388 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:32.798259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df599145-25e3-474e-82f5-81dcfef4a388 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:32.800512 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df599145-25e3-474e-82f5-81dcfef4a388 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:32.809699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 713/1394] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:22:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:22:34.232491 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aaaa58-feee-4cee-81a0-8aa7502566bc None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:34.232960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aaaa58-feee-4cee-81a0-8aa7502566bc None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:34.232960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aaaa58-feee-4cee-81a0-8aa7502566bc None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:34.234327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 682/1395] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:34.240360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad173f-fc35-43eb-b0ba-5e478e32fa5b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:34.240945 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad173f-fc35-43eb-b0ba-5e478e32fa5b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:34.240945 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad173f-fc35-43eb-b0ba-5e478e32fa5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:34.258509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-3aad173f-fc35-43eb-b0ba-5e478e32fa5b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:34.259704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3aad173f-fc35-43eb-b0ba-5e478e32fa5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:34.267449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 714/1396] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:22:34.323001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae623f-af30-4562-aeaf-a12885294315 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:34.323370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae623f-af30-4562-aeaf-a12885294315 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:34.323676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae623f-af30-4562-aeaf-a12885294315 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:34.325044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 683/1397] 10.222.0.25 () {54 vars in 962 bytes} [Mon May 9 18:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:34.330088 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc00a2-e7f8-45ab-97d5-46f369b875a9 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:34.330552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc00a2-e7f8-45ab-97d5-46f369b875a9 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:34.330898 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc00a2-e7f8-45ab-97d5-46f369b875a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:34.377639 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-41bc00a2-e7f8-45ab-97d5-46f369b875a9 None None] MFA Rules not processed for user `77cf284e81ba4042b66c0bc189f1ae2b`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:34.378806 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-41bc00a2-e7f8-45ab-97d5-46f369b875a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:34.436445 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 715/1398] 10.222.0.25 () {58 vars in 1059 bytes} [Mon May 9 18:22:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:34.444040 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae623f-af30-4562-aeaf-a12885294315 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:34.454330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae623f-af30-4562-aeaf-a12885294315 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:34.459125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b90ff-8494-4132-ac7b-07af96f40410 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cf284e81ba4042b66c0bc189f1ae2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:34.459820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b90ff-8494-4132-ac7b-07af96f40410 service glance] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:34.460104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b90ff-8494-4132-ac7b-07af96f40410 service glance] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:34.460379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b90ff-8494-4132-ac7b-07af96f40410 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:34.469256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b90ff-8494-4132-ac7b-07af96f40410 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:34.470668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b90ff-8494-4132-ac7b-07af96f40410 service glance] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:34.480417 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 684/1399] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:22:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:22:34.891118 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a3742-3894-4e70-80d4-fb46ced62c99 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:34.893748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a3742-3894-4e70-80d4-fb46ced62c99 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:34.894040 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a3742-3894-4e70-80d4-fb46ced62c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:34.940847 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-642a3742-3894-4e70-80d4-fb46ced62c99 None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:34.942020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-642a3742-3894-4e70-80d4-fb46ced62c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:35.003396 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 716/1400] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:22:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:35.046725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7b8cb-64a6-40d4-a6d0-e4032e72eebc None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:35.047025 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7b8cb-64a6-40d4-a6d0-e4032e72eebc None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:35.047340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7b8cb-64a6-40d4-a6d0-e4032e72eebc None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:35.048575 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2388c-b830-48cc-9ea8-6743b0f42d24 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:35.048715 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 685/1401] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:35.049147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2388c-b830-48cc-9ea8-6743b0f42d24 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:35.049967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2388c-b830-48cc-9ea8-6743b0f42d24 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:35.051344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 717/1402] 10.222.0.25 () {54 vars in 942 bytes} [Mon May 9 18:22:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:22:35.054372 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8d306-1f7f-411e-bb4b-c608e227042c None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:35.054782 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8d306-1f7f-411e-bb4b-c608e227042c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:35.054926 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8d306-1f7f-411e-bb4b-c608e227042c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:35.058003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11570d52-ba6f-4ba8-8851-cf00f93162c9 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:35.058389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11570d52-ba6f-4ba8-8851-cf00f93162c9 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:35.058759 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11570d52-ba6f-4ba8-8851-cf00f93162c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:35.072170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-c7d8d306-1f7f-411e-bb4b-c608e227042c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:35.073277 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c7d8d306-1f7f-411e-bb4b-c608e227042c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:35.081278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 686/1403] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:35.105954 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-11570d52-ba6f-4ba8-8851-cf00f93162c9 None None] MFA Rules not processed for user `ec2db3aa2b7d41f89314fec960d898b7`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:35.107135 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-11570d52-ba6f-4ba8-8851-cf00f93162c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:35.125397 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8d306-1f7f-411e-bb4b-c608e227042c None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:35.137175 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8d306-1f7f-411e-bb4b-c608e227042c None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:35.142871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9fc6b-5afb-46da-bdcf-6058c8c7b6f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:35.143551 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9fc6b-5afb-46da-bdcf-6058c8c7b6f6 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:35.143831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9fc6b-5afb-46da-bdcf-6058c8c7b6f6 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:35.144106 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9fc6b-5afb-46da-bdcf-6058c8c7b6f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:35.152033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9fc6b-5afb-46da-bdcf-6058c8c7b6f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:35.153593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9fc6b-5afb-46da-bdcf-6058c8c7b6f6 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:35.163984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 687/1404] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:22:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:22:35.178374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 718/1405] 10.222.0.25 () {58 vars in 1039 bytes} [Mon May 9 18:22:35 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:22:35.188182 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9fc6b-5afb-46da-bdcf-6058c8c7b6f6 service nova] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:35.198680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9fc6b-5afb-46da-bdcf-6058c8c7b6f6 service nova] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:35.203681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c260e-7192-4be3-9bd0-39108a2708e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:35.204299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c260e-7192-4be3-9bd0-39108a2708e3 service swift] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:35.204579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c260e-7192-4be3-9bd0-39108a2708e3 service swift] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:35.204851 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c260e-7192-4be3-9bd0-39108a2708e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:35.213708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c260e-7192-4be3-9bd0-39108a2708e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:35.214932 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c260e-7192-4be3-9bd0-39108a2708e3 service swift] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:35.224181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 688/1406] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:22:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 18:22:37.448377 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5665eb-e57e-4c25-8c6d-2b89aa6bac8b None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:37.448940 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5665eb-e57e-4c25-8c6d-2b89aa6bac8b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:37.448940 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5665eb-e57e-4c25-8c6d-2b89aa6bac8b None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:37.453746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 719/1407] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:22:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:37.455640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c50e2-1262-43a5-b223-1440f0a93f1f None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:37.456177 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c50e2-1262-43a5-b223-1440f0a93f1f None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:37.456381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c50e2-1262-43a5-b223-1440f0a93f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:37.476812 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ad5c50e2-1262-43a5-b223-1440f0a93f1f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:37.478047 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ad5c50e2-1262-43a5-b223-1440f0a93f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:37.486394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 689/1408] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:22:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:37.531034 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5665eb-e57e-4c25-8c6d-2b89aa6bac8b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:22:37.548236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5665eb-e57e-4c25-8c6d-2b89aa6bac8b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:22:37.554920 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c106b-1d39-4617-852c-1d0300e4adf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:22:37.555748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c106b-1d39-4617-852c-1d0300e4adf0 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:37.556100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c106b-1d39-4617-852c-1d0300e4adf0 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:37.556450 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c106b-1d39-4617-852c-1d0300e4adf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:37.564893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c106b-1d39-4617-852c-1d0300e4adf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:22:37.566457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c106b-1d39-4617-852c-1d0300e4adf0 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:22:37.577643 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 720/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:22:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:22:58.808199 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858a2ba-d480-432f-8007-f13e6248dd5e None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:58.808837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858a2ba-d480-432f-8007-f13e6248dd5e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:58.808837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858a2ba-d480-432f-8007-f13e6248dd5e None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:58.810124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 690/1410] 10.222.0.25 () {54 vars in 918 bytes} [Mon May 9 18:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:58.816109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c29c5-2f35-4ffc-849c-b43e69b3067a None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:58.816588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c29c5-2f35-4ffc-849c-b43e69b3067a None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:58.816693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c29c5-2f35-4ffc-849c-b43e69b3067a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:58.833919 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2a9c29c5-2f35-4ffc-849c-b43e69b3067a None None] MFA Rules not processed for user `f747dcb955fa499a972c46587d3db294`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:58.835094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2a9c29c5-2f35-4ffc-849c-b43e69b3067a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:58.843440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 721/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Mon May 9 18:22:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:22:58.857728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0cec4-9912-4ff6-b678-abb91b478b85 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:58.858017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0cec4-9912-4ff6-b678-abb91b478b85 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:58.858294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0cec4-9912-4ff6-b678-abb91b478b85 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:58.859665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 691/1412] 10.222.0.25 () {54 vars in 918 bytes} [Mon May 9 18:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:22:58.864808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3925a33-a17b-4c06-a76f-c2f9631a376e None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:22:58.865093 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3925a33-a17b-4c06-a76f-c2f9631a376e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:22:58.865369 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3925a33-a17b-4c06-a76f-c2f9631a376e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:22:58.882725 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b3925a33-a17b-4c06-a76f-c2f9631a376e None None] MFA Rules not processed for user `f769ee9f31354e5296645fb1e3bd84b6`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:22:58.883926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b3925a33-a17b-4c06-a76f-c2f9631a376e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:22:58.893544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 722/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Mon May 9 18:22:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:01.550553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324acd75-8167-4abb-b088-a09fbd13af01 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:01.551134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324acd75-8167-4abb-b088-a09fbd13af01 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:01.551134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324acd75-8167-4abb-b088-a09fbd13af01 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:01.552487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 692/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:01.557925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21650e32-711a-40c6-97c8-5410f3730363 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:01.558564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21650e32-711a-40c6-97c8-5410f3730363 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:01.558564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21650e32-711a-40c6-97c8-5410f3730363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:01.575858 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-21650e32-711a-40c6-97c8-5410f3730363 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:01.577028 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-21650e32-711a-40c6-97c8-5410f3730363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:01.585333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 723/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:01.626474 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324acd75-8167-4abb-b088-a09fbd13af01 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:01.640139 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324acd75-8167-4abb-b088-a09fbd13af01 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:01.645888 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65024909-f2d5-44b9-ae9d-021ed90cc243 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:01.646514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65024909-f2d5-44b9-ae9d-021ed90cc243 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:01.646795 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65024909-f2d5-44b9-ae9d-021ed90cc243 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:01.647073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65024909-f2d5-44b9-ae9d-021ed90cc243 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:01.654880 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65024909-f2d5-44b9-ae9d-021ed90cc243 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:01.656460 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65024909-f2d5-44b9-ae9d-021ed90cc243 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:01.666017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 693/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:02.644775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6229b-397b-4e4a-8fdd-571156591a36 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:02.645349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6229b-397b-4e4a-8fdd-571156591a36 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:02.645349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6229b-397b-4e4a-8fdd-571156591a36 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:02.646733 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 724/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 18:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:02.651614 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601abd7-b457-4a0a-98a5-a50d41b1e3a8 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:02.652041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601abd7-b457-4a0a-98a5-a50d41b1e3a8 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:02.652150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601abd7-b457-4a0a-98a5-a50d41b1e3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:02.669941 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-d601abd7-b457-4a0a-98a5-a50d41b1e3a8 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:02.671107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-d601abd7-b457-4a0a-98a5-a50d41b1e3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:02.680315 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 694/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 18:23:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:02.836521 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6229b-397b-4e4a-8fdd-571156591a36 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:02.850845 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6229b-397b-4e4a-8fdd-571156591a36 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:02.856144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e277e-6216-4fc4-853a-6c9f0e357fee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:02.856767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e277e-6216-4fc4-853a-6c9f0e357fee service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:02.857046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e277e-6216-4fc4-853a-6c9f0e357fee service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:02.857319 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e277e-6216-4fc4-853a-6c9f0e357fee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:02.865779 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e277e-6216-4fc4-853a-6c9f0e357fee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:02.867374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e277e-6216-4fc4-853a-6c9f0e357fee service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:02.876654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 725/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:03.807191 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0a2db-9aa7-48bf-81c4-97b5551640a3 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:03.807757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0a2db-9aa7-48bf-81c4-97b5551640a3 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:03.807757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0a2db-9aa7-48bf-81c4-97b5551640a3 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:03.809113 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 695/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 18:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:03.814261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c3592-0803-4ec7-9d78-14844b055afd None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:03.814664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c3592-0803-4ec7-9d78-14844b055afd None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:03.814843 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c3592-0803-4ec7-9d78-14844b055afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:03.832619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-543c3592-0803-4ec7-9d78-14844b055afd None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:03.833785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-543c3592-0803-4ec7-9d78-14844b055afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:03.841433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 726/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 18:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:03.989021 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0a2db-9aa7-48bf-81c4-97b5551640a3 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:04.004041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0a2db-9aa7-48bf-81c4-97b5551640a3 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:04.009416 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ace33-87b9-4e21-9439-db2c1c44b227 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:04.010059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ace33-87b9-4e21-9439-db2c1c44b227 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:04.010344 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ace33-87b9-4e21-9439-db2c1c44b227 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:04.010621 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ace33-87b9-4e21-9439-db2c1c44b227 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:04.018495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ace33-87b9-4e21-9439-db2c1c44b227 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:04.020078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ace33-87b9-4e21-9439-db2c1c44b227 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:04.029171 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 696/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:04.945423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9d6d7-591e-4804-aba5-e1d09a98cfd8 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:04.946019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9d6d7-591e-4804-aba5-e1d09a98cfd8 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:04.946019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9d6d7-591e-4804-aba5-e1d09a98cfd8 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:04.947412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 727/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 18:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:04.952189 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8afd1b-6f08-4b24-a8cb-5ceac35cf725 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:04.952620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8afd1b-6f08-4b24-a8cb-5ceac35cf725 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:04.952773 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8afd1b-6f08-4b24-a8cb-5ceac35cf725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:04.971546 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ca8afd1b-6f08-4b24-a8cb-5ceac35cf725 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:04.972723 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ca8afd1b-6f08-4b24-a8cb-5ceac35cf725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:04.980352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 697/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 18:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:05.125814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9d6d7-591e-4804-aba5-e1d09a98cfd8 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:05.140280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9d6d7-591e-4804-aba5-e1d09a98cfd8 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:05.146246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14ae6e-e0f1-4a33-b74f-fef4cba7ee8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:05.146876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14ae6e-e0f1-4a33-b74f-fef4cba7ee8d service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:05.147160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14ae6e-e0f1-4a33-b74f-fef4cba7ee8d service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:05.147467 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14ae6e-e0f1-4a33-b74f-fef4cba7ee8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:05.155363 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14ae6e-e0f1-4a33-b74f-fef4cba7ee8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:05.156928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14ae6e-e0f1-4a33-b74f-fef4cba7ee8d service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:05.166045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 728/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:06.082158 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c547cbf-2f9a-45f6-bbe6-f571c7c2b43b None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:06.082720 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c547cbf-2f9a-45f6-bbe6-f571c7c2b43b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:06.082720 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c547cbf-2f9a-45f6-bbe6-f571c7c2b43b None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:06.084124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 698/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 18:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:06.089058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d56061-dbea-43fe-a42a-5927f1c72bb0 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:06.089497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d56061-dbea-43fe-a42a-5927f1c72bb0 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:06.089595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d56061-dbea-43fe-a42a-5927f1c72bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:06.107685 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-60d56061-dbea-43fe-a42a-5927f1c72bb0 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:06.108860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-60d56061-dbea-43fe-a42a-5927f1c72bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:06.116566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 729/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 18:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:06.264808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c547cbf-2f9a-45f6-bbe6-f571c7c2b43b None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:06.279515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c547cbf-2f9a-45f6-bbe6-f571c7c2b43b None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:06.285412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e290cf6-905e-41ce-889c-4a65627f9948 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:06.286052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e290cf6-905e-41ce-889c-4a65627f9948 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:06.286336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e290cf6-905e-41ce-889c-4a65627f9948 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:06.286613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e290cf6-905e-41ce-889c-4a65627f9948 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:06.294458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e290cf6-905e-41ce-889c-4a65627f9948 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:06.296059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e290cf6-905e-41ce-889c-4a65627f9948 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:06.305143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 699/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:07.169221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bab655-e95a-4ab4-a5e2-92677cb12dd2 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:07.169794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bab655-e95a-4ab4-a5e2-92677cb12dd2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:07.169794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bab655-e95a-4ab4-a5e2-92677cb12dd2 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:07.171118 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 730/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 18:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:07.176197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcce0b-21bd-41cc-b381-ec5cb61a4333 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:07.176628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcce0b-21bd-41cc-b381-ec5cb61a4333 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:07.176783 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcce0b-21bd-41cc-b381-ec5cb61a4333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:07.195505 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-c4dcce0b-21bd-41cc-b381-ec5cb61a4333 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:07.196690 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c4dcce0b-21bd-41cc-b381-ec5cb61a4333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:07.204353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 700/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 18:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:23:07.359876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bab655-e95a-4ab4-a5e2-92677cb12dd2 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:07.373880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bab655-e95a-4ab4-a5e2-92677cb12dd2 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:07.380036 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d6092-a0f2-4c07-9f4f-767d8c01cda5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:07.380700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d6092-a0f2-4c07-9f4f-767d8c01cda5 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:07.380982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d6092-a0f2-4c07-9f4f-767d8c01cda5 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:07.381256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d6092-a0f2-4c07-9f4f-767d8c01cda5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:07.389250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d6092-a0f2-4c07-9f4f-767d8c01cda5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:07.390811 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d6092-a0f2-4c07-9f4f-767d8c01cda5 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:07.400572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 731/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:08.266010 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0429d-0747-4611-aaac-1504b0a57813 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:08.266437 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0429d-0747-4611-aaac-1504b0a57813 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:08.266568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0429d-0747-4611-aaac-1504b0a57813 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:08.267966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 701/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 18:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:08.272937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28386642-801a-468f-af93-e3b195f14c32 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:08.273333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28386642-801a-468f-af93-e3b195f14c32 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:08.273486 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28386642-801a-468f-af93-e3b195f14c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:08.290764 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-28386642-801a-468f-af93-e3b195f14c32 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:08.291949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-28386642-801a-468f-af93-e3b195f14c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:08.299490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 732/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 18:23:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:08.450438 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0429d-0747-4611-aaac-1504b0a57813 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:08.466555 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0429d-0747-4611-aaac-1504b0a57813 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:08.472054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc705a98-ea6a-4dfc-86d0-2d0a42b2a7ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:08.472635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc705a98-ea6a-4dfc-86d0-2d0a42b2a7ba service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:08.472917 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc705a98-ea6a-4dfc-86d0-2d0a42b2a7ba service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:08.473193 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc705a98-ea6a-4dfc-86d0-2d0a42b2a7ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:08.481057 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc705a98-ea6a-4dfc-86d0-2d0a42b2a7ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:08.482620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc705a98-ea6a-4dfc-86d0-2d0a42b2a7ba service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:08.492234 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 702/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:09.357350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44480b-bc17-45ce-8c8a-7f8cfcfef6ed None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:09.357921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44480b-bc17-45ce-8c8a-7f8cfcfef6ed None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:09.357921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44480b-bc17-45ce-8c8a-7f8cfcfef6ed None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:09.359299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 733/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 18:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:09.364899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec910ff8-0122-4457-8c3e-a09ca66270e1 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:09.365287 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec910ff8-0122-4457-8c3e-a09ca66270e1 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:09.365455 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec910ff8-0122-4457-8c3e-a09ca66270e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:09.383193 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ec910ff8-0122-4457-8c3e-a09ca66270e1 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:09.384378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ec910ff8-0122-4457-8c3e-a09ca66270e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:09.392105 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 703/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 18:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:09.538843 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44480b-bc17-45ce-8c8a-7f8cfcfef6ed None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:09.553142 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44480b-bc17-45ce-8c8a-7f8cfcfef6ed None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:09.558344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b73f31-a426-4729-8dba-53ee8fa88632 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:09.558965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b73f31-a426-4729-8dba-53ee8fa88632 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:09.559277 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b73f31-a426-4729-8dba-53ee8fa88632 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:09.559551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b73f31-a426-4729-8dba-53ee8fa88632 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:09.567282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b73f31-a426-4729-8dba-53ee8fa88632 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:09.568850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b73f31-a426-4729-8dba-53ee8fa88632 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:09.577815 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 734/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:10.444466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fa776-d9e8-4b85-a698-2d425060f489 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:10.445004 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fa776-d9e8-4b85-a698-2d425060f489 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:10.445004 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fa776-d9e8-4b85-a698-2d425060f489 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:10.446383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 704/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 18:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:10.451211 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d788a-32a5-4fb7-88f2-2b49aee932cf None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:10.451664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d788a-32a5-4fb7-88f2-2b49aee932cf None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:10.451768 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d788a-32a5-4fb7-88f2-2b49aee932cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:10.469766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-f52d788a-32a5-4fb7-88f2-2b49aee932cf None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:10.470997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f52d788a-32a5-4fb7-88f2-2b49aee932cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:10.478682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 735/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 18:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:10.628562 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fa776-d9e8-4b85-a698-2d425060f489 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:10.642587 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fa776-d9e8-4b85-a698-2d425060f489 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:10.647885 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486da05c-c9eb-4309-bae8-52d7456460c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:10.648518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486da05c-c9eb-4309-bae8-52d7456460c1 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:10.648804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486da05c-c9eb-4309-bae8-52d7456460c1 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:10.649078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486da05c-c9eb-4309-bae8-52d7456460c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:10.656894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486da05c-c9eb-4309-bae8-52d7456460c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:10.658445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486da05c-c9eb-4309-bae8-52d7456460c1 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:10.667603 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 705/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:12.732644 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52231395-fee4-40fb-8aee-19824c9be70d None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:12.733190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52231395-fee4-40fb-8aee-19824c9be70d None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:12.733190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52231395-fee4-40fb-8aee-19824c9be70d None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:12.734533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 736/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:12.740432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2e326-6792-418d-b642-d39bf959e014 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:12.740818 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2e326-6792-418d-b642-d39bf959e014 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:12.740991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2e326-6792-418d-b642-d39bf959e014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:12.758924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-8ae2e326-6792-418d-b642-d39bf959e014 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:12.760297 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-8ae2e326-6792-418d-b642-d39bf959e014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:12.768090 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 706/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:12.806393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52231395-fee4-40fb-8aee-19824c9be70d None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:12.816621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52231395-fee4-40fb-8aee-19824c9be70d None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:12.820036 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5a31f-099c-4b8c-a23c-bdae88416fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:12.820656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a31f-099c-4b8c-a23c-bdae88416fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:12.820930 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a31f-099c-4b8c-a23c-bdae88416fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:12.821197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a31f-099c-4b8c-a23c-bdae88416fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:12.829704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5a31f-099c-4b8c-a23c-bdae88416fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:12.831637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5a31f-099c-4b8c-a23c-bdae88416fa0 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:12.840534 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 737/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:15.221495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ffb8f-89c6-4bbe-98f4-f042867cce9e None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:15.222030 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ffb8f-89c6-4bbe-98f4-f042867cce9e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:15.222030 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ffb8f-89c6-4bbe-98f4-f042867cce9e None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:15.223469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 707/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:15.228941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19306fa-c959-48ce-8d37-a01ab9afb468 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:15.229388 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19306fa-c959-48ce-8d37-a01ab9afb468 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:15.229486 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19306fa-c959-48ce-8d37-a01ab9afb468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:15.247020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-a19306fa-c959-48ce-8d37-a01ab9afb468 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:15.248220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a19306fa-c959-48ce-8d37-a01ab9afb468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:15.256741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 738/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:15.293935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ffb8f-89c6-4bbe-98f4-f042867cce9e None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:15.295640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c94ffb8f-89c6-4bbe-98f4-f042867cce9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:15.308495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ffb8f-89c6-4bbe-98f4-f042867cce9e None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:15.312131 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741abac-be21-4bc4-a3b9-acaa6cda1c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:15.312754 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741abac-be21-4bc4-a3b9-acaa6cda1c9e service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:15.313032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741abac-be21-4bc4-a3b9-acaa6cda1c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:15.313301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741abac-be21-4bc4-a3b9-acaa6cda1c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:15.323021 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741abac-be21-4bc4-a3b9-acaa6cda1c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:15.324473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741abac-be21-4bc4-a3b9-acaa6cda1c9e service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:15.333625 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 708/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:17.913090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32170090-f5c1-452e-8d0f-524bc41c27e6 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:17.913617 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32170090-f5c1-452e-8d0f-524bc41c27e6 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:17.913617 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32170090-f5c1-452e-8d0f-524bc41c27e6 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:17.914920 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 739/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:17.920374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e57f0-fb84-42c6-965a-e4f018ac6db0 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:17.920785 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e57f0-fb84-42c6-965a-e4f018ac6db0 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:17.920933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e57f0-fb84-42c6-965a-e4f018ac6db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:17.937997 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-b29e57f0-fb84-42c6-965a-e4f018ac6db0 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:17.939163 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b29e57f0-fb84-42c6-965a-e4f018ac6db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:17.947357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 709/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:23:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:17.984615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32170090-f5c1-452e-8d0f-524bc41c27e6 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:17.995590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32170090-f5c1-452e-8d0f-524bc41c27e6 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:17.999054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c928aa9-5b2f-4a9b-be24-c9d8d7cee0e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:17.999958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c928aa9-5b2f-4a9b-be24-c9d8d7cee0e2 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:18.000243 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c928aa9-5b2f-4a9b-be24-c9d8d7cee0e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:18.000550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c928aa9-5b2f-4a9b-be24-c9d8d7cee0e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:18.010235 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c928aa9-5b2f-4a9b-be24-c9d8d7cee0e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:18.011755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c928aa9-5b2f-4a9b-be24-c9d8d7cee0e2 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:18.022018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 740/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:20.375426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e9496-7203-4986-bfa3-fb981da4dc61 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:20.376036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e9496-7203-4986-bfa3-fb981da4dc61 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:20.376036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e9496-7203-4986-bfa3-fb981da4dc61 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:20.377325 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 710/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:20.382553 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ef74b-641d-4d52-9425-50e3d86137bc None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:20.382973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ef74b-641d-4d52-9425-50e3d86137bc None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:20.383086 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ef74b-641d-4d52-9425-50e3d86137bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:20.400497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2f5ef74b-641d-4d52-9425-50e3d86137bc None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:20.401650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2f5ef74b-641d-4d52-9425-50e3d86137bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:20.409172 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 741/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:23:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:20.446540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e9496-7203-4986-bfa3-fb981da4dc61 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:20.457234 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e9496-7203-4986-bfa3-fb981da4dc61 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:20.460668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504604c4-6836-4022-9c0b-ac86ff16769d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:20.461290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504604c4-6836-4022-9c0b-ac86ff16769d service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:20.461572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504604c4-6836-4022-9c0b-ac86ff16769d service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:20.461841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504604c4-6836-4022-9c0b-ac86ff16769d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:20.470292 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504604c4-6836-4022-9c0b-ac86ff16769d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:20.472349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504604c4-6836-4022-9c0b-ac86ff16769d service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:20.481605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 711/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:23:22.790104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ff225-b5b0-4b94-bd93-8b35c408ba70 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:22.790705 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ff225-b5b0-4b94-bd93-8b35c408ba70 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:22.790705 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ff225-b5b0-4b94-bd93-8b35c408ba70 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:22.792053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 742/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:22.797433 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716df81b-8cd5-4254-a6d0-4f494fb5e478 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:22.797864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716df81b-8cd5-4254-a6d0-4f494fb5e478 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:22.797972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716df81b-8cd5-4254-a6d0-4f494fb5e478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:22.815061 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-716df81b-8cd5-4254-a6d0-4f494fb5e478 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:22.816313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-716df81b-8cd5-4254-a6d0-4f494fb5e478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:22.824206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 712/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:23:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:22.863346 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ff225-b5b0-4b94-bd93-8b35c408ba70 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:22.873710 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ff225-b5b0-4b94-bd93-8b35c408ba70 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:22.877405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ef939-ea04-4bfa-8be8-71146994ce4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:22.878531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ef939-ea04-4bfa-8be8-71146994ce4d service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:22.878806 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ef939-ea04-4bfa-8be8-71146994ce4d service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:22.879079 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ef939-ea04-4bfa-8be8-71146994ce4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:22.887717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ef939-ea04-4bfa-8be8-71146994ce4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:22.889121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ef939-ea04-4bfa-8be8-71146994ce4d service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:22.898805 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 743/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:25.140311 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ce489-5839-4ceb-a3df-2ca2a75ac59d None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:25.140894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ce489-5839-4ceb-a3df-2ca2a75ac59d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:25.140894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ce489-5839-4ceb-a3df-2ca2a75ac59d None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:25.142255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 713/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:25.147736 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1d075-1010-496e-9631-2829469c4857 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:25.148201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1d075-1010-496e-9631-2829469c4857 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:25.148305 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1d075-1010-496e-9631-2829469c4857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:25.165917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4bf1d075-1010-496e-9631-2829469c4857 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:25.167086 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4bf1d075-1010-496e-9631-2829469c4857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:25.175569 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 744/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:25.212107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ce489-5839-4ceb-a3df-2ca2a75ac59d None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:25.222819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ce489-5839-4ceb-a3df-2ca2a75ac59d None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:25.226285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d03de-d2fb-42ba-b5d3-6f6c6de39245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:25.226938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d03de-d2fb-42ba-b5d3-6f6c6de39245 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:25.227247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d03de-d2fb-42ba-b5d3-6f6c6de39245 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:25.227519 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d03de-d2fb-42ba-b5d3-6f6c6de39245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:25.235972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d03de-d2fb-42ba-b5d3-6f6c6de39245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:25.237963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d03de-d2fb-42ba-b5d3-6f6c6de39245 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:25.246929 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 714/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:27.510159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e63d3-0529-4dcf-b7e7-8f70a53fba38 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:27.510728 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e63d3-0529-4dcf-b7e7-8f70a53fba38 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:27.510728 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e63d3-0529-4dcf-b7e7-8f70a53fba38 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:27.512110 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 745/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:27.517429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b74a83-c6f4-431b-b6c3-8e72846e5e22 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:27.517867 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b74a83-c6f4-431b-b6c3-8e72846e5e22 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:27.517973 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b74a83-c6f4-431b-b6c3-8e72846e5e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:27.535751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-e5b74a83-c6f4-431b-b6c3-8e72846e5e22 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:27.536929 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e5b74a83-c6f4-431b-b6c3-8e72846e5e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:27.544660 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 715/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:23:27.582110 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e63d3-0529-4dcf-b7e7-8f70a53fba38 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:27.592599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e63d3-0529-4dcf-b7e7-8f70a53fba38 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:27.596094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69731b0-d457-4b1d-88e1-52ef243dd0a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:27.596754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69731b0-d457-4b1d-88e1-52ef243dd0a0 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:27.597039 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69731b0-d457-4b1d-88e1-52ef243dd0a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:27.597317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69731b0-d457-4b1d-88e1-52ef243dd0a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:27.606125 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69731b0-d457-4b1d-88e1-52ef243dd0a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:27.607584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69731b0-d457-4b1d-88e1-52ef243dd0a0 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:27.616696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 746/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:29.864402 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79ef6a-3ab1-4b82-97a8-93dadecbfead None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:29.864991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79ef6a-3ab1-4b82-97a8-93dadecbfead None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:29.864991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79ef6a-3ab1-4b82-97a8-93dadecbfead None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:29.866404 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 716/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:29.873721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890aead-b251-4fab-ac59-9ee1503f7f4d None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:29.874491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890aead-b251-4fab-ac59-9ee1503f7f4d None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:29.874491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890aead-b251-4fab-ac59-9ee1503f7f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:29.893217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2890aead-b251-4fab-ac59-9ee1503f7f4d None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:29.895065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2890aead-b251-4fab-ac59-9ee1503f7f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:29.902870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 747/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:29.938408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79ef6a-3ab1-4b82-97a8-93dadecbfead None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:29.949545 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79ef6a-3ab1-4b82-97a8-93dadecbfead None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:29.953107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a6892-9703-4619-b53a-6d471ac0b003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:29.953737 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a6892-9703-4619-b53a-6d471ac0b003 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:29.954017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a6892-9703-4619-b53a-6d471ac0b003 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:29.954289 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a6892-9703-4619-b53a-6d471ac0b003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:29.963965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a6892-9703-4619-b53a-6d471ac0b003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:29.965448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a6892-9703-4619-b53a-6d471ac0b003 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:29.974757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 717/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:32.222861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8bb40-6341-47e1-8071-896af69cb795 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:32.223436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8bb40-6341-47e1-8071-896af69cb795 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:32.223436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8bb40-6341-47e1-8071-896af69cb795 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:32.224717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 748/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:32.230054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43dcfe-573f-4c2b-9f1b-97997872fa19 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:32.230478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43dcfe-573f-4c2b-9f1b-97997872fa19 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:32.230631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43dcfe-573f-4c2b-9f1b-97997872fa19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:32.248697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-de43dcfe-573f-4c2b-9f1b-97997872fa19 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:32.250129 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-de43dcfe-573f-4c2b-9f1b-97997872fa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:32.258507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 718/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:23:32.293956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8bb40-6341-47e1-8071-896af69cb795 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:32.304859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8bb40-6341-47e1-8071-896af69cb795 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:32.308330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0026426-41fe-40d3-9777-639c95fc744a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:32.308957 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0026426-41fe-40d3-9777-639c95fc744a service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:32.309267 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0026426-41fe-40d3-9777-639c95fc744a service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:32.309587 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0026426-41fe-40d3-9777-639c95fc744a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:32.318951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0026426-41fe-40d3-9777-639c95fc744a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:32.320397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0026426-41fe-40d3-9777-639c95fc744a service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:32.329484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 749/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:23:34.530192 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c864af-fec1-406d-a873-69469d1ea353 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:34.530763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c864af-fec1-406d-a873-69469d1ea353 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:34.530763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c864af-fec1-406d-a873-69469d1ea353 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:34.532129 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 719/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:23:34.537963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eff442-1e07-4d9f-97bc-f084f56220cd None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:34.538393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eff442-1e07-4d9f-97bc-f084f56220cd None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:34.538511 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eff442-1e07-4d9f-97bc-f084f56220cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:34.555588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-21eff442-1e07-4d9f-97bc-f084f56220cd None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:23:34.556748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-21eff442-1e07-4d9f-97bc-f084f56220cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:23:34.564332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 750/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:23:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:23:34.600643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c864af-fec1-406d-a873-69469d1ea353 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:23:34.611037 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c864af-fec1-406d-a873-69469d1ea353 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:23:34.614541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ccf69-81c1-4a09-9278-8d198d082264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:23:34.615170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ccf69-81c1-4a09-9278-8d198d082264 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:23:34.615490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ccf69-81c1-4a09-9278-8d198d082264 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:23:34.615773 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ccf69-81c1-4a09-9278-8d198d082264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:23:34.624333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ccf69-81c1-4a09-9278-8d198d082264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:23:34.626019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ccf69-81c1-4a09-9278-8d198d082264 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:23:34.635100 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 720/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:23:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:24:07.673746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a3eb7-4c9e-464a-9dbc-77c1c9344721 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:07.674348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a3eb7-4c9e-464a-9dbc-77c1c9344721 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:07.674348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a3eb7-4c9e-464a-9dbc-77c1c9344721 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:07.676260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 751/1471] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:24:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:24:07.681430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf774e-4c30-468f-a981-75c869e866d1 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:07.681944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf774e-4c30-468f-a981-75c869e866d1 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:07.682054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf774e-4c30-468f-a981-75c869e866d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:07.699945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-2aaf774e-4c30-468f-a981-75c869e866d1 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:24:07.701112 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-2aaf774e-4c30-468f-a981-75c869e866d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:24:07.708856 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 721/1472] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:24:07.755496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed03c4-eb83-41eb-aad6-e7493eee2f11 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:07.755782 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed03c4-eb83-41eb-aad6-e7493eee2f11 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:07.756060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed03c4-eb83-41eb-aad6-e7493eee2f11 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:07.757406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 752/1473] 10.222.0.25 () {54 vars in 962 bytes} [Mon May 9 18:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:24:07.762293 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0a853-49c9-43be-a91b-7bc02a4afe6d None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:07.762584 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0a853-49c9-43be-a91b-7bc02a4afe6d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:07.762866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0a853-49c9-43be-a91b-7bc02a4afe6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:07.780107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-bcf0a853-49c9-43be-a91b-7bc02a4afe6d None None] MFA Rules not processed for user `77cf284e81ba4042b66c0bc189f1ae2b`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:24:07.781267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-bcf0a853-49c9-43be-a91b-7bc02a4afe6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:24:07.787579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 722/1474] 10.222.0.25 () {58 vars in 1059 bytes} [Mon May 9 18:24:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:24:07.794547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed03c4-eb83-41eb-aad6-e7493eee2f11 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:24:07.805293 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed03c4-eb83-41eb-aad6-e7493eee2f11 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:24:07.808737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde409d-f81e-4682-a058-b71337615e5b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cf284e81ba4042b66c0bc189f1ae2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:24:07.809339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde409d-f81e-4682-a058-b71337615e5b service glance] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:07.809612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde409d-f81e-4682-a058-b71337615e5b service glance] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:07.809880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde409d-f81e-4682-a058-b71337615e5b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:07.818891 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde409d-f81e-4682-a058-b71337615e5b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:24:07.820319 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde409d-f81e-4682-a058-b71337615e5b service glance] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:24:07.829184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 753/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:24:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:24:10.101179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24393e54-11e1-4a8b-a785-1128141c7849 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:10.101742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24393e54-11e1-4a8b-a785-1128141c7849 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:10.101742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24393e54-11e1-4a8b-a785-1128141c7849 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:10.103162 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 723/1476] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:24:10.108260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a83d9c-0bc4-423f-9c24-2cbe0bde333f None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:10.108659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a83d9c-0bc4-423f-9c24-2cbe0bde333f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:10.108799 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a83d9c-0bc4-423f-9c24-2cbe0bde333f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:10.125714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-76a83d9c-0bc4-423f-9c24-2cbe0bde333f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:24:10.127120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-76a83d9c-0bc4-423f-9c24-2cbe0bde333f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:24:10.135300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 754/1477] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:24:10.158707 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24393e54-11e1-4a8b-a785-1128141c7849 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:24:10.170638 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24393e54-11e1-4a8b-a785-1128141c7849 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:24:10.175007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f21fbd-b178-4e97-8beb-356fe03f10a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:24:10.175657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f21fbd-b178-4e97-8beb-356fe03f10a1 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:10.175936 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f21fbd-b178-4e97-8beb-356fe03f10a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:10.176232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f21fbd-b178-4e97-8beb-356fe03f10a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:10.176946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f21fbd-b178-4e97-8beb-356fe03f10a1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:24:10.178665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: WARNING py.warnings [None req-02f21fbd-b178-4e97-8beb-356fe03f10a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 18:24:10.178665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: warnings.warn(msg) May 09 18:24:10.178665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: May 09 18:24:10.179819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f21fbd-b178-4e97-8beb-356fe03f10a1 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:24:10.198506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 724/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Mon May 9 18:24:10 2022] GET /identity/v3/projects => generated 2553 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:24:12.418843 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2655acd-6208-4dfa-b66b-efb6a368b45b None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:12.419420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2655acd-6208-4dfa-b66b-efb6a368b45b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:12.419420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2655acd-6208-4dfa-b66b-efb6a368b45b None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:12.420764 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 755/1479] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:24:12.425890 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8dc37-5dec-4a18-8409-f25e0aa05337 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:12.426330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8dc37-5dec-4a18-8409-f25e0aa05337 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:12.426456 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8dc37-5dec-4a18-8409-f25e0aa05337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:12.444110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-6af8dc37-5dec-4a18-8409-f25e0aa05337 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:24:12.445287 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-6af8dc37-5dec-4a18-8409-f25e0aa05337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:24:12.452839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 725/1480] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:24:12.500535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2655acd-6208-4dfa-b66b-efb6a368b45b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:24:12.515117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2655acd-6208-4dfa-b66b-efb6a368b45b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:24:12.520993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb254d7-847c-48d5-bb0d-0c367854d33b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:24:12.521620 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb254d7-847c-48d5-bb0d-0c367854d33b service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:12.521906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb254d7-847c-48d5-bb0d-0c367854d33b service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:12.522179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb254d7-847c-48d5-bb0d-0c367854d33b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:12.529971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb254d7-847c-48d5-bb0d-0c367854d33b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:24:12.531560 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb254d7-847c-48d5-bb0d-0c367854d33b service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:24:12.540567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 756/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:24:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:24:14.915702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8712ff3-bfb0-4973-b029-e9b952461c0c None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:14.916229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8712ff3-bfb0-4973-b029-e9b952461c0c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:14.916229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8712ff3-bfb0-4973-b029-e9b952461c0c None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:14.917524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 726/1482] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:24:14.922764 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef9e6-1aa0-463b-b0b7-0bedb54a6e16 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:14.923209 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef9e6-1aa0-463b-b0b7-0bedb54a6e16 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:14.923344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef9e6-1aa0-463b-b0b7-0bedb54a6e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:14.941202 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-649ef9e6-1aa0-463b-b0b7-0bedb54a6e16 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:24:14.942384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-649ef9e6-1aa0-463b-b0b7-0bedb54a6e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:24:14.950075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 757/1483] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:24:14.988484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8712ff3-bfb0-4973-b029-e9b952461c0c None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:24:14.998901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8712ff3-bfb0-4973-b029-e9b952461c0c None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:24:15.002648 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3a9d2-3201-4476-991e-17cf01da9c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:24:15.003345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3a9d2-3201-4476-991e-17cf01da9c4b service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:15.003629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3a9d2-3201-4476-991e-17cf01da9c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:15.003904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3a9d2-3201-4476-991e-17cf01da9c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:15.012483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3a9d2-3201-4476-991e-17cf01da9c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:24:15.013889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3a9d2-3201-4476-991e-17cf01da9c4b service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:24:15.023541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 727/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:24:17.298962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12ca56-981b-4efc-a661-b3a3c536d25c None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:17.299557 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12ca56-981b-4efc-a661-b3a3c536d25c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:17.299557 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12ca56-981b-4efc-a661-b3a3c536d25c None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:17.301448 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 758/1485] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:24:17.306746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7356f-640a-4bbb-838e-f4c1814b5167 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:17.307218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7356f-640a-4bbb-838e-f4c1814b5167 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:17.307332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7356f-640a-4bbb-838e-f4c1814b5167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:17.325039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-d2e7356f-640a-4bbb-838e-f4c1814b5167 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:24:17.326210 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-d2e7356f-640a-4bbb-838e-f4c1814b5167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:24:17.333901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 728/1486] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:24:17.371155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12ca56-981b-4efc-a661-b3a3c536d25c None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:24:17.382644 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12ca56-981b-4efc-a661-b3a3c536d25c None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:24:17.386166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5470b-7784-49a0-8073-8af2c1a5ad01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:24:17.386799 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5470b-7784-49a0-8073-8af2c1a5ad01 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:17.387081 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5470b-7784-49a0-8073-8af2c1a5ad01 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:17.387397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5470b-7784-49a0-8073-8af2c1a5ad01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:17.396126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa5470b-7784-49a0-8073-8af2c1a5ad01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:24:17.397529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa5470b-7784-49a0-8073-8af2c1a5ad01 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:24:17.406638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 759/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:24:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:24:19.766022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b94e2e-9fda-4814-ad70-a95bdf69644e None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:19.766885 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b94e2e-9fda-4814-ad70-a95bdf69644e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:19.766885 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b94e2e-9fda-4814-ad70-a95bdf69644e None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:19.769157 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 729/1488] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:24:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:24:19.775107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea18491-90e0-472b-8f44-d1f114ca4841 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:19.775619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea18491-90e0-472b-8f44-d1f114ca4841 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:19.775734 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea18491-90e0-472b-8f44-d1f114ca4841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:19.793329 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4ea18491-90e0-472b-8f44-d1f114ca4841 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:24:19.794487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4ea18491-90e0-472b-8f44-d1f114ca4841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:24:19.802080 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 760/1489] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:24:19.841360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b94e2e-9fda-4814-ad70-a95bdf69644e None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:24:19.852276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b94e2e-9fda-4814-ad70-a95bdf69644e None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:24:19.855748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ac557-3677-4e11-b9c5-cdf36985f27a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:24:19.856385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac557-3677-4e11-b9c5-cdf36985f27a service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:19.856665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac557-3677-4e11-b9c5-cdf36985f27a service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:19.856938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac557-3677-4e11-b9c5-cdf36985f27a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:19.865395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ac557-3677-4e11-b9c5-cdf36985f27a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:24:19.866797 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ac557-3677-4e11-b9c5-cdf36985f27a service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:24:19.875828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 730/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:24:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:24:22.326646 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c861ad-cb10-4fb1-8e08-b36273412430 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:22.327151 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c861ad-cb10-4fb1-8e08-b36273412430 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:22.327303 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c861ad-cb10-4fb1-8e08-b36273412430 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:22.328683 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 761/1491] 10.222.0.25 () {54 vars in 926 bytes} [Mon May 9 18:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:24:22.333822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de1c96-5719-4564-a1b0-6550cb12e28b None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:22.334262 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de1c96-5719-4564-a1b0-6550cb12e28b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:22.334470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de1c96-5719-4564-a1b0-6550cb12e28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:22.352844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-84de1c96-5719-4564-a1b0-6550cb12e28b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:24:22.354011 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-84de1c96-5719-4564-a1b0-6550cb12e28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:24:22.361625 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 731/1492] 10.222.0.25 () {58 vars in 1023 bytes} [Mon May 9 18:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:24:22.399495 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c861ad-cb10-4fb1-8e08-b36273412430 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:24:22.410271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c861ad-cb10-4fb1-8e08-b36273412430 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:24:22.413763 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b49fe-3b4c-4647-b066-afc7230aac4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:24:22.414390 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b49fe-3b4c-4647-b066-afc7230aac4c service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:24:22.414678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b49fe-3b4c-4647-b066-afc7230aac4c service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:24:22.414956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b49fe-3b4c-4647-b066-afc7230aac4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:24:22.423603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b49fe-3b4c-4647-b066-afc7230aac4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:24:22.425631 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b49fe-3b4c-4647-b066-afc7230aac4c service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:24:22.434879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 762/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:24:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:25:49.313340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc591d5a-fee5-4d3f-9fff-ad21caf6555c None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:49.313936 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc591d5a-fee5-4d3f-9fff-ad21caf6555c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:49.313936 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc591d5a-fee5-4d3f-9fff-ad21caf6555c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:49.332315 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-cc591d5a-fee5-4d3f-9fff-ad21caf6555c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:25:49.333477 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-cc591d5a-fee5-4d3f-9fff-ad21caf6555c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:25:49.341088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 732/1494] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:25:49.350094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b8691-1f91-4f31-869b-0f56c672d8d7 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:49.350596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b8691-1f91-4f31-869b-0f56c672d8d7 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:49.350697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b8691-1f91-4f31-869b-0f56c672d8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:49.368337 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-919b8691-1f91-4f31-869b-0f56c672d8d7 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:25:49.369510 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-919b8691-1f91-4f31-869b-0f56c672d8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:25:49.378198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 763/1495] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:25:49.382705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc591d5a-fee5-4d3f-9fff-ad21caf6555c None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:25:49.391904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc591d5a-fee5-4d3f-9fff-ad21caf6555c None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:25:49.396930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeac2b8-2e00-4631-8249-9139c87516fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:25:49.397577 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeac2b8-2e00-4631-8249-9139c87516fc admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:49.397858 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeac2b8-2e00-4631-8249-9139c87516fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:49.398133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeac2b8-2e00-4631-8249-9139c87516fc admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:49.398852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeac2b8-2e00-4631-8249-9139c87516fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:25:49.399153 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeac2b8-2e00-4631-8249-9139c87516fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:25:49.400178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: WARNING py.warnings [None req-5aeac2b8-2e00-4631-8249-9139c87516fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 18:25:49.400178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: warnings.warn(msg) May 09 18:25:49.400178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: May 09 18:25:49.401067 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeac2b8-2e00-4631-8249-9139c87516fc admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:25:49.411991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 733/1496] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:25:49.419895 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b8691-1f91-4f31-869b-0f56c672d8d7 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:25:49.428753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b8691-1f91-4f31-869b-0f56c672d8d7 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:25:49.433309 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d863a-35f9-4bd5-9d46-62f275188c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:25:49.433937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d863a-35f9-4bd5-9d46-62f275188c23 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:49.434214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d863a-35f9-4bd5-9d46-62f275188c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:49.434486 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d863a-35f9-4bd5-9d46-62f275188c23 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:49.435159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d863a-35f9-4bd5-9d46-62f275188c23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:25:49.436933 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: WARNING py.warnings [None req-5b0d863a-35f9-4bd5-9d46-62f275188c23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 18:25:49.436933 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: warnings.warn(msg) May 09 18:25:49.436933 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: May 09 18:25:49.438098 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d863a-35f9-4bd5-9d46-62f275188c23 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:25:49.473585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 764/1497] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:25:49 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:25:49.477526 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeac2b8-2e00-4631-8249-9139c87516fc admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:25:49.487705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeac2b8-2e00-4631-8249-9139c87516fc admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:25:49.541640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0662e57-e44e-4765-b91e-8b03e5b5f1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:25:49.542908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0662e57-e44e-4765-b91e-8b03e5b5f1af admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:49.542908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0662e57-e44e-4765-b91e-8b03e5b5f1af admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:49.542908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0662e57-e44e-4765-b91e-8b03e5b5f1af admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:49.543552 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0662e57-e44e-4765-b91e-8b03e5b5f1af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:25:49.544555 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: WARNING py.warnings [None req-f0662e57-e44e-4765-b91e-8b03e5b5f1af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 18:25:49.544555 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: warnings.warn(msg) May 09 18:25:49.544555 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: May 09 18:25:49.545747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0662e57-e44e-4765-b91e-8b03e5b5f1af admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:25:49.570736 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 734/1498] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:25:49 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:25:49.571899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d863a-35f9-4bd5-9d46-62f275188c23 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:25:49.584108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d863a-35f9-4bd5-9d46-62f275188c23 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:25:49.589328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2970c5-b8f9-4581-b589-cd3728a5cba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:25:49.590082 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2970c5-b8f9-4581-b589-cd3728a5cba2 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:49.590359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2970c5-b8f9-4581-b589-cd3728a5cba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:49.590635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2970c5-b8f9-4581-b589-cd3728a5cba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:49.591387 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2970c5-b8f9-4581-b589-cd3728a5cba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:25:49.592400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: WARNING py.warnings [None req-6e2970c5-b8f9-4581-b589-cd3728a5cba2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 18:25:49.592400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: warnings.warn(msg) May 09 18:25:49.592400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: May 09 18:25:49.593278 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2970c5-b8f9-4581-b589-cd3728a5cba2 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:25:49.607733 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 765/1499] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:25:49.612516 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0662e57-e44e-4765-b91e-8b03e5b5f1af admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:25:49.625967 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0662e57-e44e-4765-b91e-8b03e5b5f1af admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:25:49.634177 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db7384-da3a-4402-bfc5-7de20eea4c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:25:49.635071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db7384-da3a-4402-bfc5-7de20eea4c05 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:49.635429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db7384-da3a-4402-bfc5-7de20eea4c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:49.635768 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db7384-da3a-4402-bfc5-7de20eea4c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:49.636586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db7384-da3a-4402-bfc5-7de20eea4c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:25:49.637985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: WARNING py.warnings [None req-49db7384-da3a-4402-bfc5-7de20eea4c05 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 18:25:49.637985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: warnings.warn(msg) May 09 18:25:49.637985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: May 09 18:25:49.639135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db7384-da3a-4402-bfc5-7de20eea4c05 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:25:49.653347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 735/1500] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:25:49.657748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2970c5-b8f9-4581-b589-cd3728a5cba2 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:25:49.667807 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2970c5-b8f9-4581-b589-cd3728a5cba2 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:25:49.672314 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472c979-2976-4b54-bc3f-84ff09ad5cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:25:49.673183 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472c979-2976-4b54-bc3f-84ff09ad5cca admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:49.673461 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472c979-2976-4b54-bc3f-84ff09ad5cca admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:49.674241 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472c979-2976-4b54-bc3f-84ff09ad5cca admin admin] PATH_INFO: `/v3/projects/a279f0e98c8840528740c68aaf67227e/users/fb14c890f7d640eca3b116f58ac0db19/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:49.689234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472c979-2976-4b54-bc3f-84ff09ad5cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=a279f0e98c8840528740c68aaf67227e, user_id=fb14c890f7d640eca3b116f58ac0db19, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:25:49.690859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: WARNING py.warnings [None req-b472c979-2976-4b54-bc3f-84ff09ad5cca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 18:25:49.690859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: warnings.warn(msg) May 09 18:25:49.690859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: May 09 18:25:49.692832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472c979-2976-4b54-bc3f-84ff09ad5cca admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:25:49.705297 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 766/1501] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:25:49 2022] PUT /identity/v3/projects/a279f0e98c8840528740c68aaf67227e/users/fb14c890f7d640eca3b116f58ac0db19/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:25:49.716542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87e72a-9344-4fbe-8718-3c2f95f99aa9 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:49.716840 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87e72a-9344-4fbe-8718-3c2f95f99aa9 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:49.717116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87e72a-9344-4fbe-8718-3c2f95f99aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:49.747278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ed87e72a-9344-4fbe-8718-3c2f95f99aa9 None None] MFA Rules not processed for user `fb14c890f7d640eca3b116f58ac0db19`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:25:49.747278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ed87e72a-9344-4fbe-8718-3c2f95f99aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:25:49.810819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 736/1502] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:25:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:25:49.824670 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f1e17-93b8-4a54-97ad-c03053e7e7d5 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:49.824898 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f1e17-93b8-4a54-97ad-c03053e7e7d5 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:49.825177 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f1e17-93b8-4a54-97ad-c03053e7e7d5 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:49.826568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 767/1503] 10.222.0.25 () {54 vars in 963 bytes} [Mon May 9 18:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:25:49.831318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e99b-efbc-43e5-b291-e43858209825 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:49.831606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e99b-efbc-43e5-b291-e43858209825 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:49.831881 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e99b-efbc-43e5-b291-e43858209825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:49.849524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-5cc6e99b-efbc-43e5-b291-e43858209825 None None] MFA Rules not processed for user `4bda352a10f241ca96d5308a55238d57`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:25:49.850678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-5cc6e99b-efbc-43e5-b291-e43858209825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:25:49.900878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 737/1504] 10.222.0.25 () {58 vars in 1060 bytes} [Mon May 9 18:25:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:25:49.909089 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f1e17-93b8-4a54-97ad-c03053e7e7d5 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:25:49.919323 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f1e17-93b8-4a54-97ad-c03053e7e7d5 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:25:49.922768 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4602d91-5a3e-40c6-aebd-8ca16233543d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:25:49.923436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4602d91-5a3e-40c6-aebd-8ca16233543d service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:49.923722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4602d91-5a3e-40c6-aebd-8ca16233543d service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:49.924653 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4602d91-5a3e-40c6-aebd-8ca16233543d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:49.932620 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4602d91-5a3e-40c6-aebd-8ca16233543d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:25:49.934018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4602d91-5a3e-40c6-aebd-8ca16233543d service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:25:49.942594 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 768/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:25:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:25:49.985001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fddb3-b9db-470e-86ac-a09e3823e47e None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:49.985288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fddb3-b9db-470e-86ac-a09e3823e47e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:49.985561 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fddb3-b9db-470e-86ac-a09e3823e47e None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:49.987506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 738/1506] 10.222.0.25 () {52 vars in 829 bytes} [Mon May 9 18:25:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:25:49.991131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4602d91-5a3e-40c6-aebd-8ca16233543d service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:25:50.051578 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4602d91-5a3e-40c6-aebd-8ca16233543d service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:25:50.055997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:25:50.056860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:50.057142 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:50.057418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] PATH_INFO: `/v3/users/fb14c890f7d640eca3b116f58ac0db19` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:50.058570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb14c890f7d640eca3b116f58ac0db19)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:25:50.059602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: WARNING py.warnings [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 18:25:50.059602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: warnings.warn(msg) May 09 18:25:50.059602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: May 09 18:25:50.060799 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:25:50.122391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb14c890f7d640eca3b116f58ac0db19'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:25:50.123057 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb14c890f7d640eca3b116f58ac0db19'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:25:50.132610 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb14c890f7d640eca3b116f58ac0db19'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:25:50.144214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb14c890f7d640eca3b116f58ac0db19'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:25:50.144871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb14c890f7d640eca3b116f58ac0db19'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:25:50.168938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 769/1507] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:25:49 2022] DELETE /identity/v3/users/fb14c890f7d640eca3b116f58ac0db19 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:25:50.173452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6dc82-9e83-4db7-8154-47db3baa0e31 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:50.173747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6dc82-9e83-4db7-8154-47db3baa0e31 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:50.174568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6dc82-9e83-4db7-8154-47db3baa0e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:50.191353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-2be6dc82-9e83-4db7-8154-47db3baa0e31 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:25:50.192525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-2be6dc82-9e83-4db7-8154-47db3baa0e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:25:50.247365 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 739/1508] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:25:50.257044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:25:50.258399 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:25:50.316130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e321e-129d-42ad-af34-461a782ecc1a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:25:50.319705 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f0d07-dd0a-4911-961c-c4e415896e3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:25:50.320343 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f0d07-dd0a-4911-961c-c4e415896e3e service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:50.320623 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f0d07-dd0a-4911-961c-c4e415896e3e service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:50.320889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f0d07-dd0a-4911-961c-c4e415896e3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:50.329357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f0d07-dd0a-4911-961c-c4e415896e3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:25:50.331336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f0d07-dd0a-4911-961c-c4e415896e3e service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:25:50.340268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 770/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:25:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:25:50.715663 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6dc82-9e83-4db7-8154-47db3baa0e31 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:25:50.717135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-2be6dc82-9e83-4db7-8154-47db3baa0e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:25:50.729318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6dc82-9e83-4db7-8154-47db3baa0e31 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:25:50.735596 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06251138-3799-4042-a811-7a129aabe303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:25:50.737062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06251138-3799-4042-a811-7a129aabe303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:25:50.737414 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06251138-3799-4042-a811-7a129aabe303 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:25:50.737769 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06251138-3799-4042-a811-7a129aabe303 admin admin] PATH_INFO: `/v3/projects/a279f0e98c8840528740c68aaf67227e` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:25:50.739031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06251138-3799-4042-a811-7a129aabe303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a279f0e98c8840528740c68aaf67227e)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:25:50.741699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: WARNING py.warnings [None req-06251138-3799-4042-a811-7a129aabe303 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 18:25:50.741699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: warnings.warn(msg) May 09 18:25:50.741699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: May 09 18:25:50.742023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06251138-3799-4042-a811-7a129aabe303 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:25:50.805868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-06251138-3799-4042-a811-7a129aabe303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a279f0e98c8840528740c68aaf67227e', 'request_id': 'req-06251138-3799-4042-a811-7a129aabe303'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:25:50.812585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-06251138-3799-4042-a811-7a129aabe303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a279f0e98c8840528740c68aaf67227e', 'request_id': 'req-06251138-3799-4042-a811-7a129aabe303'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:25:50.823581 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-06251138-3799-4042-a811-7a129aabe303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a279f0e98c8840528740c68aaf67227e', 'request_id': 'req-06251138-3799-4042-a811-7a129aabe303'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:25:50.844027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-06251138-3799-4042-a811-7a129aabe303 admin admin] The token cache is being invalidate because project a279f0e98c8840528740c68aaf67227e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:25:50.844317 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-06251138-3799-4042-a811-7a129aabe303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:25:50.844968 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-06251138-3799-4042-a811-7a129aabe303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:25:50.846627 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 740/1510] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:25:50 2022] DELETE /identity/v3/projects/a279f0e98c8840528740c68aaf67227e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:27:34.220429 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57836f79-4825-4e65-addc-f212ff069178 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:34.220981 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57836f79-4825-4e65-addc-f212ff069178 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:34.220981 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57836f79-4825-4e65-addc-f212ff069178 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:34.222336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 771/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 18:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:27:34.227125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06932071-a060-44e0-828a-6efe91afe32f None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:34.227598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06932071-a060-44e0-828a-6efe91afe32f None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:34.227793 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06932071-a060-44e0-828a-6efe91afe32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:34.245812 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-06932071-a060-44e0-828a-6efe91afe32f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:27:34.246986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-06932071-a060-44e0-828a-6efe91afe32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:27:34.302627 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 741/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 18:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:27:34.465763 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57836f79-4825-4e65-addc-f212ff069178 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:27:34.467426 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-57836f79-4825-4e65-addc-f212ff069178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:27:34.539439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57836f79-4825-4e65-addc-f212ff069178 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:27:34.544796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f4fa9-3991-4eaf-ac40-b7d4a9e5b9a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:27:34.545435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f4fa9-3991-4eaf-ac40-b7d4a9e5b9a7 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:34.545717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f4fa9-3991-4eaf-ac40-b7d4a9e5b9a7 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:34.545995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f4fa9-3991-4eaf-ac40-b7d4a9e5b9a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:34.554612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f4fa9-3991-4eaf-ac40-b7d4a9e5b9a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:27:34.556219 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f4fa9-3991-4eaf-ac40-b7d4a9e5b9a7 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:27:34.565365 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 772/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:27:35.158019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0588e9-7de0-431d-9142-c71fc28cd41e None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:35.158312 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0588e9-7de0-431d-9142-c71fc28cd41e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:35.158594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0588e9-7de0-431d-9142-c71fc28cd41e None None] PATH_INFO: `/v3` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:35.160147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 742/1514] 10.222.0.36 () {54 vars in 925 bytes} [Mon May 9 18:27:35 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:27:35.168018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ff0e3-c562-4c78-9852-482c753dc1ce None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:35.168307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ff0e3-c562-4c78-9852-482c753dc1ce None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:35.168586 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ff0e3-c562-4c78-9852-482c753dc1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:35.186555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-007ff0e3-c562-4c78-9852-482c753dc1ce None None] MFA Rules not processed for user `f747dcb955fa499a972c46587d3db294`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:27:35.187839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-007ff0e3-c562-4c78-9852-482c753dc1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:27:35.246480 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 773/1515] 10.222.0.36 () {58 vars in 1015 bytes} [Mon May 9 18:27:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:27:35.458497 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e200f9-b657-4236-8b14-373a5d40270f None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:35.459036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e200f9-b657-4236-8b14-373a5d40270f None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:35.459036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e200f9-b657-4236-8b14-373a5d40270f None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:35.460475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 743/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 18:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:27:35.465254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb311d5-6f4a-4c8e-9356-ddc6a9648a73 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:35.465542 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb311d5-6f4a-4c8e-9356-ddc6a9648a73 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:35.465820 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb311d5-6f4a-4c8e-9356-ddc6a9648a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:35.483338 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-abb311d5-6f4a-4c8e-9356-ddc6a9648a73 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:27:35.484512 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-abb311d5-6f4a-4c8e-9356-ddc6a9648a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:27:35.492143 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 774/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 18:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:27:35.642267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e200f9-b657-4236-8b14-373a5d40270f None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:27:35.656824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e200f9-b657-4236-8b14-373a5d40270f None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:27:35.662473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695e95c-5af3-4622-8a70-5199c25e049e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:27:35.663117 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695e95c-5af3-4622-8a70-5199c25e049e service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:35.663431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695e95c-5af3-4622-8a70-5199c25e049e service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:35.663705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695e95c-5af3-4622-8a70-5199c25e049e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:35.671975 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695e95c-5af3-4622-8a70-5199c25e049e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:27:35.674095 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695e95c-5af3-4622-8a70-5199c25e049e service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:27:35.683189 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 744/1518] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:27:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:27:45.329761 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db329a-40e0-4aa7-b6c5-1b112bbe984b None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:45.330952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db329a-40e0-4aa7-b6c5-1b112bbe984b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:45.330952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db329a-40e0-4aa7-b6c5-1b112bbe984b None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:45.332269 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 775/1519] 10.222.0.25 () {54 vars in 960 bytes} [Mon May 9 18:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:27:45.337364 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb708c-990c-44d1-8aeb-5b23099d6f76 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:45.337807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb708c-990c-44d1-8aeb-5b23099d6f76 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:45.337969 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb708c-990c-44d1-8aeb-5b23099d6f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:45.356096 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-d9eb708c-990c-44d1-8aeb-5b23099d6f76 None None] MFA Rules not processed for user `f747dcb955fa499a972c46587d3db294`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:27:45.357270 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-d9eb708c-990c-44d1-8aeb-5b23099d6f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:27:45.365559 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 745/1520] 10.222.0.25 () {58 vars in 1057 bytes} [Mon May 9 18:27:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:27:45.372958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db329a-40e0-4aa7-b6c5-1b112bbe984b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:27:45.386211 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db329a-40e0-4aa7-b6c5-1b112bbe984b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:27:45.390950 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1d1fc-273d-46a1-b1bd-7f0bf7544ea6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f747dcb955fa499a972c46587d3db294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:27:45.391607 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1d1fc-273d-46a1-b1bd-7f0bf7544ea6 service placement] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:45.391993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1d1fc-273d-46a1-b1bd-7f0bf7544ea6 service placement] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:45.392265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1d1fc-273d-46a1-b1bd-7f0bf7544ea6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:45.400738 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1d1fc-273d-46a1-b1bd-7f0bf7544ea6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:27:45.402306 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1d1fc-273d-46a1-b1bd-7f0bf7544ea6 service placement] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:27:45.412202 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 776/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:27:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:27:46.989982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cd3ac-b912-4a36-952e-60eb106d9b02 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:46.990470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cd3ac-b912-4a36-952e-60eb106d9b02 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:46.990600 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cd3ac-b912-4a36-952e-60eb106d9b02 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:46.992000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 746/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 18:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 18:27:46.996762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f07a7-6f71-45eb-b31d-117989250963 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:46.997171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f07a7-6f71-45eb-b31d-117989250963 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:46.997315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f07a7-6f71-45eb-b31d-117989250963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:47.014968 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-fe8f07a7-6f71-45eb-b31d-117989250963 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:27:47.016166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-fe8f07a7-6f71-45eb-b31d-117989250963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:27:47.024643 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 777/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 18:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:27:47.179110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cd3ac-b912-4a36-952e-60eb106d9b02 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:27:47.193554 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cd3ac-b912-4a36-952e-60eb106d9b02 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:27:47.198726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb452576-b5d2-4079-99cb-07235b26adbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:27:47.199396 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb452576-b5d2-4079-99cb-07235b26adbf service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:47.199676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb452576-b5d2-4079-99cb-07235b26adbf service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:47.199945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb452576-b5d2-4079-99cb-07235b26adbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:47.208333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb452576-b5d2-4079-99cb-07235b26adbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:27:47.209898 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb452576-b5d2-4079-99cb-07235b26adbf service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:27:47.219295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 747/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:27:48.110216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69413-6c83-4484-9aa0-7ef0a5c9177d None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:48.110814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69413-6c83-4484-9aa0-7ef0a5c9177d None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:48.110814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69413-6c83-4484-9aa0-7ef0a5c9177d None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:48.112769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 778/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 18:27:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:27:48.117480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ec851-423c-4e65-83b5-33c728a46a92 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:48.117895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ec851-423c-4e65-83b5-33c728a46a92 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:48.118096 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ec851-423c-4e65-83b5-33c728a46a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:48.136000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-533ec851-423c-4e65-83b5-33c728a46a92 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:27:48.137181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-533ec851-423c-4e65-83b5-33c728a46a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:27:48.151825 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 748/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 18:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:27:48.308548 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf69413-6c83-4484-9aa0-7ef0a5c9177d None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:27:48.322836 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf69413-6c83-4484-9aa0-7ef0a5c9177d None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:27:48.328117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e30154-08eb-499f-b7d8-836d9876296e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:27:48.328747 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30154-08eb-499f-b7d8-836d9876296e service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:48.329026 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30154-08eb-499f-b7d8-836d9876296e service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:48.329300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30154-08eb-499f-b7d8-836d9876296e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:48.337894 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e30154-08eb-499f-b7d8-836d9876296e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:27:48.339446 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e30154-08eb-499f-b7d8-836d9876296e service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:27:48.349226 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 779/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:27:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:27:52.273522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036302e5-78af-46f6-b0c1-aa34e5a84224 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:52.274485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036302e5-78af-46f6-b0c1-aa34e5a84224 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:52.274485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036302e5-78af-46f6-b0c1-aa34e5a84224 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:52.275504 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 749/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:27:52.280631 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94360b52-313e-49fe-80df-7292f40a871c None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:52.281090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94360b52-313e-49fe-80df-7292f40a871c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:52.281187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94360b52-313e-49fe-80df-7292f40a871c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:52.300084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-94360b52-313e-49fe-80df-7292f40a871c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:27:52.301284 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-94360b52-313e-49fe-80df-7292f40a871c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:27:52.309238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 780/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:27:52.344777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036302e5-78af-46f6-b0c1-aa34e5a84224 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:27:52.346430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-036302e5-78af-46f6-b0c1-aa34e5a84224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:27:52.408097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036302e5-78af-46f6-b0c1-aa34e5a84224 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:27:52.411650 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea08c7-4470-4770-98a5-a9899f77ba09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:27:52.412299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea08c7-4470-4770-98a5-a9899f77ba09 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:52.412581 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea08c7-4470-4770-98a5-a9899f77ba09 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:52.412862 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea08c7-4470-4770-98a5-a9899f77ba09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:52.422284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea08c7-4470-4770-98a5-a9899f77ba09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:27:52.424314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea08c7-4470-4770-98a5-a9899f77ba09 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:27:52.434056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 750/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:27:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:27:54.679663 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42bab3-15f6-4a88-a3e9-cc0ed30bba84 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:54.680236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42bab3-15f6-4a88-a3e9-cc0ed30bba84 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:54.680236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42bab3-15f6-4a88-a3e9-cc0ed30bba84 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:54.681495 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 781/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:27:54.691351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372df20-9ea1-4aef-a4c1-bb6ed09fceff None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:54.691840 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372df20-9ea1-4aef-a4c1-bb6ed09fceff None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:54.691939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372df20-9ea1-4aef-a4c1-bb6ed09fceff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:54.710939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-a372df20-9ea1-4aef-a4c1-bb6ed09fceff None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:27:54.712150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a372df20-9ea1-4aef-a4c1-bb6ed09fceff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:27:54.719796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 751/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:27:54.755558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42bab3-15f6-4a88-a3e9-cc0ed30bba84 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:27:54.766008 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42bab3-15f6-4a88-a3e9-cc0ed30bba84 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:27:54.769733 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e5a3e-9676-4da4-880c-ae8571948f11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:27:54.770365 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e5a3e-9676-4da4-880c-ae8571948f11 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:54.770642 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e5a3e-9676-4da4-880c-ae8571948f11 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:54.770914 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e5a3e-9676-4da4-880c-ae8571948f11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:54.779678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e5a3e-9676-4da4-880c-ae8571948f11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:27:54.781100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e5a3e-9676-4da4-880c-ae8571948f11 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:27:54.790735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 782/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:27:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:27:58.511391 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2349566-ba0f-40d7-8a49-914dfde7eac7 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:58.511988 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2349566-ba0f-40d7-8a49-914dfde7eac7 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:58.511988 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2349566-ba0f-40d7-8a49-914dfde7eac7 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:58.513313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 752/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:27:58.519028 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49b293-e7a7-4f2c-8efb-d8d04dd9f1fe None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:58.519470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49b293-e7a7-4f2c-8efb-d8d04dd9f1fe None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:58.519623 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49b293-e7a7-4f2c-8efb-d8d04dd9f1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:58.536802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-1b49b293-e7a7-4f2c-8efb-d8d04dd9f1fe None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:27:58.537962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-1b49b293-e7a7-4f2c-8efb-d8d04dd9f1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:27:58.545630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 783/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:27:58.816499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2349566-ba0f-40d7-8a49-914dfde7eac7 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:27:58.818128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a2349566-ba0f-40d7-8a49-914dfde7eac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:27:58.872608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2349566-ba0f-40d7-8a49-914dfde7eac7 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:27:58.876110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d28dc-43b7-4077-b4b7-6e50f1d0363b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:27:58.877319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d28dc-43b7-4077-b4b7-6e50f1d0363b service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:27:58.877602 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d28dc-43b7-4077-b4b7-6e50f1d0363b service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:27:58.877877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d28dc-43b7-4077-b4b7-6e50f1d0363b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:27:58.886817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d28dc-43b7-4077-b4b7-6e50f1d0363b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:27:58.888264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d28dc-43b7-4077-b4b7-6e50f1d0363b service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:27:58.897313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 753/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:28:01.168615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09074509-4a77-4c97-ac2a-dfadd80416c8 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:28:01.169187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09074509-4a77-4c97-ac2a-dfadd80416c8 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:28:01.169187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09074509-4a77-4c97-ac2a-dfadd80416c8 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:28:01.170531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 784/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 18:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:28:01.176397 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314747e-ba63-4f56-8556-8efb0841689b None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:28:01.176827 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314747e-ba63-4f56-8556-8efb0841689b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:28:01.176959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314747e-ba63-4f56-8556-8efb0841689b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:28:01.195095 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-a314747e-ba63-4f56-8556-8efb0841689b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:28:01.196310 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a314747e-ba63-4f56-8556-8efb0841689b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:28:01.203992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 754/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 18:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:28:01.483277 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09074509-4a77-4c97-ac2a-dfadd80416c8 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:28:01.485043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-09074509-4a77-4c97-ac2a-dfadd80416c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:28:01.498247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09074509-4a77-4c97-ac2a-dfadd80416c8 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:28:01.501745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3be62-dc12-429d-b422-d80ea6babd12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:28:01.502377 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3be62-dc12-429d-b422-d80ea6babd12 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:28:01.502659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3be62-dc12-429d-b422-d80ea6babd12 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:28:01.502937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3be62-dc12-429d-b422-d80ea6babd12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:28:01.512091 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3be62-dc12-429d-b422-d80ea6babd12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:28:01.513525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3be62-dc12-429d-b422-d80ea6babd12 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:28:01.524301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 785/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:28:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:31.192348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9352f26-f39f-409f-81d0-719bc67d2d1d None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.193006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9352f26-f39f-409f-81d0-719bc67d2d1d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.193006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9352f26-f39f-409f-81d0-719bc67d2d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.225469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-a9352f26-f39f-409f-81d0-719bc67d2d1d None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.226607 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a9352f26-f39f-409f-81d0-719bc67d2d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.238553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 755/1540] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:31.249661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cdde7-1422-4600-a2eb-bd1b45ce8eac None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.250256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cdde7-1422-4600-a2eb-bd1b45ce8eac None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.250256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cdde7-1422-4600-a2eb-bd1b45ce8eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.276492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-bc9cdde7-1422-4600-a2eb-bd1b45ce8eac None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.277926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-bc9cdde7-1422-4600-a2eb-bd1b45ce8eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.290029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243c5dd-a708-4fe9-ab48-86945841a487 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.290338 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243c5dd-a708-4fe9-ab48-86945841a487 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.290594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243c5dd-a708-4fe9-ab48-86945841a487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.290721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 786/1541] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:31.295369 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cdde7-1422-4600-a2eb-bd1b45ce8eac None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.304780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cdde7-1422-4600-a2eb-bd1b45ce8eac None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.314763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-7243c5dd-a708-4fe9-ab48-86945841a487 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.315993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7243c5dd-a708-4fe9-ab48-86945841a487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.319279 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21f089-235b-4677-9eb7-dad504224df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.319980 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21f089-235b-4677-9eb7-dad504224df7 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.320264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21f089-235b-4677-9eb7-dad504224df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.320539 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21f089-235b-4677-9eb7-dad504224df7 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.321282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21f089-235b-4677-9eb7-dad504224df7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:29:31.321748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21f089-235b-4677-9eb7-dad504224df7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.322796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: WARNING py.warnings [None req-8c21f089-235b-4677-9eb7-dad504224df7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 18:29:31.322796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: warnings.warn(msg) May 09 18:29:31.322796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: May 09 18:29:31.323833 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21f089-235b-4677-9eb7-dad504224df7 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.335899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 756/1542] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:29:31.342545 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 787/1543] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:29:31.351519 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243c5dd-a708-4fe9-ab48-86945841a487 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.353494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde19e6-6a45-4d6a-98f2-3fcff17a9f2a None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.353789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde19e6-6a45-4d6a-98f2-3fcff17a9f2a None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.354072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde19e6-6a45-4d6a-98f2-3fcff17a9f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.371651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243c5dd-a708-4fe9-ab48-86945841a487 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.385427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-ffde19e6-6a45-4d6a-98f2-3fcff17a9f2a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.387268 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d07f7-b49f-42ae-9385-addf0b1d3e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.387509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ffde19e6-6a45-4d6a-98f2-3fcff17a9f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.387620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d07f7-b49f-42ae-9385-addf0b1d3e27 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.387620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d07f7-b49f-42ae-9385-addf0b1d3e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.387620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d07f7-b49f-42ae-9385-addf0b1d3e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.388412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d07f7-b49f-42ae-9385-addf0b1d3e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.389492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: WARNING py.warnings [None req-bd3d07f7-b49f-42ae-9385-addf0b1d3e27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 18:29:31.389492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: warnings.warn(msg) May 09 18:29:31.389492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: May 09 18:29:31.395346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d07f7-b49f-42ae-9385-addf0b1d3e27 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.406800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 788/1544] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:29:31.411647 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde19e6-6a45-4d6a-98f2-3fcff17a9f2a None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.426075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde19e6-6a45-4d6a-98f2-3fcff17a9f2a None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.439772 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f5441-8d96-4dd4-ac91-29007b763a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.440455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f5441-8d96-4dd4-ac91-29007b763a49 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.440736 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f5441-8d96-4dd4-ac91-29007b763a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.441007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f5441-8d96-4dd4-ac91-29007b763a49 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.441744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f5441-8d96-4dd4-ac91-29007b763a49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:29:31.442200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f5441-8d96-4dd4-ac91-29007b763a49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.443688 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f5441-8d96-4dd4-ac91-29007b763a49 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.445165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 757/1545] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/projects => generated 393 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:31.446674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccd0d9-cc99-4bc2-9161-1cf39148e2ed None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.446973 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccd0d9-cc99-4bc2-9161-1cf39148e2ed None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.447294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccd0d9-cc99-4bc2-9161-1cf39148e2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.457910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 789/1546] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:29:31.458739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f5441-8d96-4dd4-ac91-29007b763a49 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.473589 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f5441-8d96-4dd4-ac91-29007b763a49 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.482798 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-efccd0d9-cc99-4bc2-9161-1cf39148e2ed None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.483992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-efccd0d9-cc99-4bc2-9161-1cf39148e2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.550890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00704941-0c8f-4e59-ac32-634524f3888b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.551594 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00704941-0c8f-4e59-ac32-634524f3888b admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.551880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00704941-0c8f-4e59-ac32-634524f3888b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.552158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00704941-0c8f-4e59-ac32-634524f3888b admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.552835 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00704941-0c8f-4e59-ac32-634524f3888b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.553848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: WARNING py.warnings [None req-00704941-0c8f-4e59-ac32-634524f3888b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 18:29:31.553848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: warnings.warn(msg) May 09 18:29:31.553848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: May 09 18:29:31.555084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00704941-0c8f-4e59-ac32-634524f3888b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.561046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 758/1547] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:29:31.563115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705c73b-3e1b-4fa0-95ea-596e2c871cb4 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.563353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705c73b-3e1b-4fa0-95ea-596e2c871cb4 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.563590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705c73b-3e1b-4fa0-95ea-596e2c871cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.576425 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 790/1548] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:31.578541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bb2cb-a6ed-4f77-93bf-afc77e9a4140 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.578738 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bb2cb-a6ed-4f77-93bf-afc77e9a4140 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.579016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bb2cb-a6ed-4f77-93bf-afc77e9a4140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.583280 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-b705c73b-3e1b-4fa0-95ea-596e2c871cb4 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.584681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b705c73b-3e1b-4fa0-95ea-596e2c871cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.592594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 759/1549] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:31.593500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705c73b-3e1b-4fa0-95ea-596e2c871cb4 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.599434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-7e7bb2cb-a6ed-4f77-93bf-afc77e9a4140 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.600465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7e7bb2cb-a6ed-4f77-93bf-afc77e9a4140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.602738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705c73b-3e1b-4fa0-95ea-596e2c871cb4 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.607135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc55431-c451-4cdc-bd0b-4ca0bb379970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.607815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc55431-c451-4cdc-bd0b-4ca0bb379970 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.608266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc55431-c451-4cdc-bd0b-4ca0bb379970 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.608704 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc55431-c451-4cdc-bd0b-4ca0bb379970 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.609079 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 791/1550] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:31.609450 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc55431-c451-4cdc-bd0b-4ca0bb379970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.610432 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406a19a-c608-4758-83d8-6ab2d107c71c None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.610725 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406a19a-c608-4758-83d8-6ab2d107c71c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.611002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406a19a-c608-4758-83d8-6ab2d107c71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.611549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc55431-c451-4cdc-bd0b-4ca0bb379970 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.630943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9406a19a-c608-4758-83d8-6ab2d107c71c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.632163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9406a19a-c608-4758-83d8-6ab2d107c71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.639124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 760/1551] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:31.640796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 792/1552] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:31.641108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ddf58-8676-42c4-aa96-05187535e6ea None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.641404 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ddf58-8676-42c4-aa96-05187535e6ea None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.641685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ddf58-8676-42c4-aa96-05187535e6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.641919 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62ffb5-b24f-4aaa-9a54-d0fc3704830a None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.642184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62ffb5-b24f-4aaa-9a54-d0fc3704830a None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.642542 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62ffb5-b24f-4aaa-9a54-d0fc3704830a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.663797 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f71ddf58-8676-42c4-aa96-05187535e6ea None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.664168 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9b62ffb5-b24f-4aaa-9a54-d0fc3704830a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.664983 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f71ddf58-8676-42c4-aa96-05187535e6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.665318 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9b62ffb5-b24f-4aaa-9a54-d0fc3704830a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.723134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 793/1553] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:31.723531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 761/1554] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:31.724598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ddf58-8676-42c4-aa96-05187535e6ea None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.724723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a1f4e-70f8-482f-abba-151886a2e06e None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.724890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a1f4e-70f8-482f-abba-151886a2e06e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.725175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a1f4e-70f8-482f-abba-151886a2e06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.741103 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ddf58-8676-42c4-aa96-05187535e6ea None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.745722 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57631e9-a1d1-49f5-9826-4079ad40a85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.746385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57631e9-a1d1-49f5-9826-4079ad40a85f admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.746667 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57631e9-a1d1-49f5-9826-4079ad40a85f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.746951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57631e9-a1d1-49f5-9826-4079ad40a85f admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.747705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57631e9-a1d1-49f5-9826-4079ad40a85f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.748194 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-876a1f4e-70f8-482f-abba-151886a2e06e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.749295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57631e9-a1d1-49f5-9826-4079ad40a85f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.749400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-876a1f4e-70f8-482f-abba-151886a2e06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.757893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 794/1555] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:31.762855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 762/1556] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:31.763007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67851ceb-f53b-4772-a4ca-e6282f6546b5 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.763007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67851ceb-f53b-4772-a4ca-e6282f6546b5 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.763007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67851ceb-f53b-4772-a4ca-e6282f6546b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.763274 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaff925-9ed0-46fc-9649-ee414439d24e None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.763274 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaff925-9ed0-46fc-9649-ee414439d24e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.763274 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaff925-9ed0-46fc-9649-ee414439d24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.779464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-67851ceb-f53b-4772-a4ca-e6282f6546b5 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.779878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-beaff925-9ed0-46fc-9649-ee414439d24e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.780640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-67851ceb-f53b-4772-a4ca-e6282f6546b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.781053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-beaff925-9ed0-46fc-9649-ee414439d24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.788409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 795/1557] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:31.791928 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 763/1558] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:31.792066 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67851ceb-f53b-4772-a4ca-e6282f6546b5 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.792175 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e9c58-a0f5-4095-b404-38c24a0b3d3d None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.792175 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e9c58-a0f5-4095-b404-38c24a0b3d3d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.792175 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e9c58-a0f5-4095-b404-38c24a0b3d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.801041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67851ceb-f53b-4772-a4ca-e6282f6546b5 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.805640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136bd916-44f4-42cd-9a88-1b83a09ae764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.806320 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bd916-44f4-42cd-9a88-1b83a09ae764 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.806601 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bd916-44f4-42cd-9a88-1b83a09ae764 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.806871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bd916-44f4-42cd-9a88-1b83a09ae764 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.807580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136bd916-44f4-42cd-9a88-1b83a09ae764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.808408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f01e9c58-a0f5-4095-b404-38c24a0b3d3d None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.809467 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136bd916-44f4-42cd-9a88-1b83a09ae764 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.809571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f01e9c58-a0f5-4095-b404-38c24a0b3d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.817667 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 764/1559] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:31.821265 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d9654-d6c1-4b5c-969e-82239f5a7a39 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.821265 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d9654-d6c1-4b5c-969e-82239f5a7a39 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.821265 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d9654-d6c1-4b5c-969e-82239f5a7a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.835357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 796/1560] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:31.835357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e7e9e-8496-4267-affa-073e3e036c09 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.835357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e7e9e-8496-4267-affa-073e3e036c09 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.835357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e7e9e-8496-4267-affa-073e3e036c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.838887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-182d9654-d6c1-4b5c-969e-82239f5a7a39 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.840074 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-182d9654-d6c1-4b5c-969e-82239f5a7a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.848169 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 765/1561] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:31.848978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d9654-d6c1-4b5c-969e-82239f5a7a39 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.851675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-553e7e9e-8496-4267-affa-073e3e036c09 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:31.852892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-553e7e9e-8496-4267-affa-073e3e036c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:31.861111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d9654-d6c1-4b5c-969e-82239f5a7a39 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.861259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 797/1562] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:31.861983 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e7e9e-8496-4267-affa-073e3e036c09 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.865781 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422a017-c2cb-4402-8fce-f9f419908085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.866432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422a017-c2cb-4402-8fce-f9f419908085 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.866898 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422a017-c2cb-4402-8fce-f9f419908085 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.867190 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422a017-c2cb-4402-8fce-f9f419908085 admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.867918 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422a017-c2cb-4402-8fce-f9f419908085 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:29:31.868226 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422a017-c2cb-4402-8fce-f9f419908085 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.869808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422a017-c2cb-4402-8fce-f9f419908085 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.871265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e7e9e-8496-4267-affa-073e3e036c09 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.876252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64516a1b-b117-409e-b78a-955622e35ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.876881 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64516a1b-b117-409e-b78a-955622e35ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.877160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64516a1b-b117-409e-b78a-955622e35ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.877434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64516a1b-b117-409e-b78a-955622e35ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.878397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64516a1b-b117-409e-b78a-955622e35ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.879978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64516a1b-b117-409e-b78a-955622e35ae6 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.882196 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 766/1563] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:29:31.884097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422a017-c2cb-4402-8fce-f9f419908085 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.892684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 798/1564] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:31.892684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64516a1b-b117-409e-b78a-955622e35ae6 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.894048 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422a017-c2cb-4402-8fce-f9f419908085 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.899242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9757b-450e-4c29-a4aa-c9d4f3de827a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.899880 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9757b-450e-4c29-a4aa-c9d4f3de827a admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.900001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64516a1b-b117-409e-b78a-955622e35ae6 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.900159 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9757b-450e-4c29-a4aa-c9d4f3de827a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.900438 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9757b-450e-4c29-a4aa-c9d4f3de827a admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.901143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9757b-450e-4c29-a4aa-c9d4f3de827a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:29:31.901443 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9757b-450e-4c29-a4aa-c9d4f3de827a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.902993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9757b-450e-4c29-a4aa-c9d4f3de827a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.904580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3955e-a673-433a-bd4e-3ed52a7b8ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.905242 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3955e-a673-433a-bd4e-3ed52a7b8ead admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.905522 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3955e-a673-433a-bd4e-3ed52a7b8ead admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.905844 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3955e-a673-433a-bd4e-3ed52a7b8ead admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.906547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3955e-a673-433a-bd4e-3ed52a7b8ead admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:29:31.906859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3955e-a673-433a-bd4e-3ed52a7b8ead admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.908463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3955e-a673-433a-bd4e-3ed52a7b8ead admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.914864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 767/1565] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:29:31.915695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9757b-450e-4c29-a4aa-c9d4f3de827a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.925256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9757b-450e-4c29-a4aa-c9d4f3de827a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.931246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 799/1566] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:29:31.931246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3955e-a673-433a-bd4e-3ed52a7b8ead admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.931560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef0050-bed5-4cf4-89ce-07d6c0300758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.931560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef0050-bed5-4cf4-89ce-07d6c0300758 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.931560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef0050-bed5-4cf4-89ce-07d6c0300758 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.931560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef0050-bed5-4cf4-89ce-07d6c0300758 admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.931858 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef0050-bed5-4cf4-89ce-07d6c0300758 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:29:31.931858 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef0050-bed5-4cf4-89ce-07d6c0300758 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.933602 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef0050-bed5-4cf4-89ce-07d6c0300758 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.936230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3955e-a673-433a-bd4e-3ed52a7b8ead admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.941305 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e381ef-a2e6-4563-821d-da0c3adfbb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.941943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e381ef-a2e6-4563-821d-da0c3adfbb53 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.942220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e381ef-a2e6-4563-821d-da0c3adfbb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.942499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e381ef-a2e6-4563-821d-da0c3adfbb53 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.943221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e381ef-a2e6-4563-821d-da0c3adfbb53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.944761 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e381ef-a2e6-4563-821d-da0c3adfbb53 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.945842 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 768/1567] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:29:31.946519 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef0050-bed5-4cf4-89ce-07d6c0300758 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.955619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef0050-bed5-4cf4-89ce-07d6c0300758 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.957483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 800/1568] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:31.960156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e381ef-a2e6-4563-821d-da0c3adfbb53 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.963055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5b8dd-73d7-451f-b399-21daab79203c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.963510 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5b8dd-73d7-451f-b399-21daab79203c admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.963794 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5b8dd-73d7-451f-b399-21daab79203c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.964068 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5b8dd-73d7-451f-b399-21daab79203c admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.964786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5b8dd-73d7-451f-b399-21daab79203c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:29:31.965088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5b8dd-73d7-451f-b399-21daab79203c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.966674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5b8dd-73d7-451f-b399-21daab79203c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.971811 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e381ef-a2e6-4563-821d-da0c3adfbb53 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:31.976269 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2740a3c-89da-4ef5-a3b4-0aaed25bdc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:31.976915 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2740a3c-89da-4ef5-a3b4-0aaed25bdc42 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:31.977200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2740a3c-89da-4ef5-a3b4-0aaed25bdc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:31.977478 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2740a3c-89da-4ef5-a3b4-0aaed25bdc42 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:31.978191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2740a3c-89da-4ef5-a3b4-0aaed25bdc42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:29:31.978497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2740a3c-89da-4ef5-a3b4-0aaed25bdc42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:31.983623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 769/1569] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:29:31.983623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5b8dd-73d7-451f-b399-21daab79203c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:31.983829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2740a3c-89da-4ef5-a3b4-0aaed25bdc42 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:31.999402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 801/1570] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:29:31.999402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2740a3c-89da-4ef5-a3b4-0aaed25bdc42 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.000325 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5b8dd-73d7-451f-b399-21daab79203c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.000325 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc157c-dd9c-4075-a32c-22a719afeb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.000673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc157c-dd9c-4075-a32c-22a719afeb8c admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.000955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc157c-dd9c-4075-a32c-22a719afeb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.001229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc157c-dd9c-4075-a32c-22a719afeb8c admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.001984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc157c-dd9c-4075-a32c-22a719afeb8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.003728 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2740a3c-89da-4ef5-a3b4-0aaed25bdc42 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.004134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc157c-dd9c-4075-a32c-22a719afeb8c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.008193 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542972f3-cc06-4ae0-9b7f-f6aec18c9a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.009049 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542972f3-cc06-4ae0-9b7f-f6aec18c9a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.009332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542972f3-cc06-4ae0-9b7f-f6aec18c9a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.009609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542972f3-cc06-4ae0-9b7f-f6aec18c9a3c admin admin] PATH_INFO: `/v3/projects/705eecf37fcc4fe7aabef923ece6ad59/users/acae3ad19a7e45b9877520dff8f5b995/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.024286 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542972f3-cc06-4ae0-9b7f-f6aec18c9a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=705eecf37fcc4fe7aabef923ece6ad59, user_id=acae3ad19a7e45b9877520dff8f5b995, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.026981 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542972f3-cc06-4ae0-9b7f-f6aec18c9a3c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.029110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 770/1571] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:31 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:32.029826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc157c-dd9c-4075-a32c-22a719afeb8c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.043939 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 802/1572] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:31 2022] PUT /identity/v3/projects/705eecf37fcc4fe7aabef923ece6ad59/users/acae3ad19a7e45b9877520dff8f5b995/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:32.044596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542972f3-cc06-4ae0-9b7f-f6aec18c9a3c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.091072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc157c-dd9c-4075-a32c-22a719afeb8c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.104510 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542972f3-cc06-4ae0-9b7f-f6aec18c9a3c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.109516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4858c-f057-4227-84e2-7dfd85689ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.110166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4858c-f057-4227-84e2-7dfd85689ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.110444 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4858c-f057-4227-84e2-7dfd85689ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.110717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4858c-f057-4227-84e2-7dfd85689ab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.111418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4858c-f057-4227-84e2-7dfd85689ab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.113330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4858c-f057-4227-84e2-7dfd85689ab3 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.138221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 803/1573] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:32 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:32.139146 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4858c-f057-4227-84e2-7dfd85689ab3 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.146845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a9796-6b91-48f6-a1ef-8c8d5232560b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.147558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a9796-6b91-48f6-a1ef-8c8d5232560b admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.147840 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a9796-6b91-48f6-a1ef-8c8d5232560b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.148121 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a9796-6b91-48f6-a1ef-8c8d5232560b admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.148796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a9796-6b91-48f6-a1ef-8c8d5232560b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.150681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a9796-6b91-48f6-a1ef-8c8d5232560b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.179150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 771/1574] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:32 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:32.179150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a9796-6b91-48f6-a1ef-8c8d5232560b admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.200449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4858c-f057-4227-84e2-7dfd85689ab3 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.237995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a9796-6b91-48f6-a1ef-8c8d5232560b admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.242894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f79c7-c328-4676-8821-14ded826521c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.243590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f79c7-c328-4676-8821-14ded826521c admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.243984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f79c7-c328-4676-8821-14ded826521c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.244265 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f79c7-c328-4676-8821-14ded826521c admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.244981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f79c7-c328-4676-8821-14ded826521c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.246578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f79c7-c328-4676-8821-14ded826521c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.253624 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1876562-34a0-4988-9ae4-fc060d97510a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.254223 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1876562-34a0-4988-9ae4-fc060d97510a admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.254506 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1876562-34a0-4988-9ae4-fc060d97510a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.254781 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1876562-34a0-4988-9ae4-fc060d97510a admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.255489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1876562-34a0-4988-9ae4-fc060d97510a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.256455 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 772/1575] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:32.257218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f79c7-c328-4676-8821-14ded826521c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.257395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1876562-34a0-4988-9ae4-fc060d97510a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.268288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f79c7-c328-4676-8821-14ded826521c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.273086 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc718e-9aec-4a21-9a4c-6a3e6670c92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.273793 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc718e-9aec-4a21-9a4c-6a3e6670c92a admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.274078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc718e-9aec-4a21-9a4c-6a3e6670c92a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.274414 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc718e-9aec-4a21-9a4c-6a3e6670c92a admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.275161 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc718e-9aec-4a21-9a4c-6a3e6670c92a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.277183 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc718e-9aec-4a21-9a4c-6a3e6670c92a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.286780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 804/1576] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:32 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:32.286780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1876562-34a0-4988-9ae4-fc060d97510a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.303824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 773/1577] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:32 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:32.304794 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc718e-9aec-4a21-9a4c-6a3e6670c92a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.346158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1876562-34a0-4988-9ae4-fc060d97510a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.362504 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc718e-9aec-4a21-9a4c-6a3e6670c92a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.366986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01d98b-3124-47a5-b9d1-aaa4b53090fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.367674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01d98b-3124-47a5-b9d1-aaa4b53090fb admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.367956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01d98b-3124-47a5-b9d1-aaa4b53090fb admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.368229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01d98b-3124-47a5-b9d1-aaa4b53090fb admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.368896 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01d98b-3124-47a5-b9d1-aaa4b53090fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.370795 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01d98b-3124-47a5-b9d1-aaa4b53090fb admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.391085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 774/1578] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:32 2022] POST /identity/v3/users => generated 284 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:32.391085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be527d0a-3b59-4ccb-88b2-96473ae9f557 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.391085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be527d0a-3b59-4ccb-88b2-96473ae9f557 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.391085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be527d0a-3b59-4ccb-88b2-96473ae9f557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.405028 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5f2e0-ebc2-4249-9d6c-06dd32449c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.406225 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f2e0-ebc2-4249-9d6c-06dd32449c65 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.406468 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f2e0-ebc2-4249-9d6c-06dd32449c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.406808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f2e0-ebc2-4249-9d6c-06dd32449c65 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.407551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5f2e0-ebc2-4249-9d6c-06dd32449c65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.409482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5f2e0-ebc2-4249-9d6c-06dd32449c65 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.419387 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-be527d0a-3b59-4ccb-88b2-96473ae9f557 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:32.420672 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-be527d0a-3b59-4ccb-88b2-96473ae9f557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:32.429938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 775/1579] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:32.430742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be527d0a-3b59-4ccb-88b2-96473ae9f557 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.441603 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be527d0a-3b59-4ccb-88b2-96473ae9f557 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.444243 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 805/1580] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:32 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:32.444923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5f2e0-ebc2-4249-9d6c-06dd32449c65 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.446739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc958e9-1098-48d6-9b06-19ea95902333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.447439 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc958e9-1098-48d6-9b06-19ea95902333 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.447719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc958e9-1098-48d6-9b06-19ea95902333 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.447993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc958e9-1098-48d6-9b06-19ea95902333 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.448676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc958e9-1098-48d6-9b06-19ea95902333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.450594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc958e9-1098-48d6-9b06-19ea95902333 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.473259 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 776/1581] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:32 2022] POST /identity/v3/users => generated 287 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:32.473259 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc958e9-1098-48d6-9b06-19ea95902333 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.482364 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc958e9-1098-48d6-9b06-19ea95902333 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.504420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5f2e0-ebc2-4249-9d6c-06dd32449c65 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.509330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12009f-6fd4-4bed-a4be-cca5b1ddc7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.510262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12009f-6fd4-4bed-a4be-cca5b1ddc7fb admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.510542 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12009f-6fd4-4bed-a4be-cca5b1ddc7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.510815 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12009f-6fd4-4bed-a4be-cca5b1ddc7fb admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.511522 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12009f-6fd4-4bed-a4be-cca5b1ddc7fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.513418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12009f-6fd4-4bed-a4be-cca5b1ddc7fb admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.536524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 806/1582] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:32 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:32.536524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12009f-6fd4-4bed-a4be-cca5b1ddc7fb admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.540286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76f46a-1d92-450e-a2ac-0cf65c0e9b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.541189 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76f46a-1d92-450e-a2ac-0cf65c0e9b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.541471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76f46a-1d92-450e-a2ac-0cf65c0e9b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.541747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76f46a-1d92-450e-a2ac-0cf65c0e9b1b admin admin] PATH_INFO: `/v3/projects/280795075c674e39b3a0e3c9fac9c47f/users/a9eddd862d4443e796684bbefd8264fb/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.545499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12009f-6fd4-4bed-a4be-cca5b1ddc7fb admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.550636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed10e9-0582-4741-8321-eb43e8cc665f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.551330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed10e9-0582-4741-8321-eb43e8cc665f admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.551612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed10e9-0582-4741-8321-eb43e8cc665f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.551884 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed10e9-0582-4741-8321-eb43e8cc665f admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.552557 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed10e9-0582-4741-8321-eb43e8cc665f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.554447 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed10e9-0582-4741-8321-eb43e8cc665f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.556995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76f46a-1d92-450e-a2ac-0cf65c0e9b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=280795075c674e39b3a0e3c9fac9c47f, user_id=a9eddd862d4443e796684bbefd8264fb, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.558004 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: WARNING py.warnings [None req-8c76f46a-1d92-450e-a2ac-0cf65c0e9b1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 18:29:32.558004 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: warnings.warn(msg) May 09 18:29:32.558004 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: May 09 18:29:32.559947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76f46a-1d92-450e-a2ac-0cf65c0e9b1b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.574009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 777/1583] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:32 2022] PUT /identity/v3/projects/280795075c674e39b3a0e3c9fac9c47f/users/a9eddd862d4443e796684bbefd8264fb/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:32.574786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76f46a-1d92-450e-a2ac-0cf65c0e9b1b admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.578887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 807/1584] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:32 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:32.578887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed10e9-0582-4741-8321-eb43e8cc665f admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.587771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed10e9-0582-4741-8321-eb43e8cc665f admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.635025 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76f46a-1d92-450e-a2ac-0cf65c0e9b1b admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.639557 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a54e63-2aa5-48cb-a9c6-74af647e4f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.640714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a54e63-2aa5-48cb-a9c6-74af647e4f26 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.640993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a54e63-2aa5-48cb-a9c6-74af647e4f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.641257 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a54e63-2aa5-48cb-a9c6-74af647e4f26 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.641829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc089eb-a339-40f4-bfdb-55a4f442237a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.642009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a54e63-2aa5-48cb-a9c6-74af647e4f26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.642493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc089eb-a339-40f4-bfdb-55a4f442237a admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.642773 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc089eb-a339-40f4-bfdb-55a4f442237a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.643046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc089eb-a339-40f4-bfdb-55a4f442237a admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.643808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc089eb-a339-40f4-bfdb-55a4f442237a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.643925 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a54e63-2aa5-48cb-a9c6-74af647e4f26 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.645361 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc089eb-a339-40f4-bfdb-55a4f442237a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.655424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 808/1585] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:32.656790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc089eb-a339-40f4-bfdb-55a4f442237a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.669612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 778/1586] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:32 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:32.669612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a54e63-2aa5-48cb-a9c6-74af647e4f26 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.678855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a54e63-2aa5-48cb-a9c6-74af647e4f26 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.683433 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15791b3f-5898-4b0d-b3c1-8eaf78f5e534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.684068 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791b3f-5898-4b0d-b3c1-8eaf78f5e534 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.684347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791b3f-5898-4b0d-b3c1-8eaf78f5e534 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.684621 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791b3f-5898-4b0d-b3c1-8eaf78f5e534 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.685286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15791b3f-5898-4b0d-b3c1-8eaf78f5e534 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.687309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15791b3f-5898-4b0d-b3c1-8eaf78f5e534 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.706989 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 779/1587] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:32 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:32.707779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15791b3f-5898-4b0d-b3c1-8eaf78f5e534 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.713258 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc089eb-a339-40f4-bfdb-55a4f442237a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.716854 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cff8c-5912-4fe3-8e25-3291f73e4026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.717356 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15791b3f-5898-4b0d-b3c1-8eaf78f5e534 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.717478 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cff8c-5912-4fe3-8e25-3291f73e4026 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.717765 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cff8c-5912-4fe3-8e25-3291f73e4026 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.718041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cff8c-5912-4fe3-8e25-3291f73e4026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.722375 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140cf1f-1587-411b-9404-463712818475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.723003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140cf1f-1587-411b-9404-463712818475 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.723328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140cf1f-1587-411b-9404-463712818475 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.723603 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140cf1f-1587-411b-9404-463712818475 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.724313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140cf1f-1587-411b-9404-463712818475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.725862 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140cf1f-1587-411b-9404-463712818475 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.727403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cff8c-5912-4fe3-8e25-3291f73e4026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.728804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cff8c-5912-4fe3-8e25-3291f73e4026 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.735872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 780/1588] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:32.736642 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140cf1f-1587-411b-9404-463712818475 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.738099 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 809/1589] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:32.740028 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cff8c-5912-4fe3-8e25-3291f73e4026 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.747935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140cf1f-1587-411b-9404-463712818475 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.748756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cff8c-5912-4fe3-8e25-3291f73e4026 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.752967 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3b0d8-b9bc-4ae1-ba20-2e681db03377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.753567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d720e-66f5-43a0-94c7-2bb1ed1b73a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.753696 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3b0d8-b9bc-4ae1-ba20-2e681db03377 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.753935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3b0d8-b9bc-4ae1-ba20-2e681db03377 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.754207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d720e-66f5-43a0-94c7-2bb1ed1b73a6 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.754313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3b0d8-b9bc-4ae1-ba20-2e681db03377 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.754536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d720e-66f5-43a0-94c7-2bb1ed1b73a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.754752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d720e-66f5-43a0-94c7-2bb1ed1b73a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.754922 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3b0d8-b9bc-4ae1-ba20-2e681db03377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.755493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d720e-66f5-43a0-94c7-2bb1ed1b73a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.756598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3b0d8-b9bc-4ae1-ba20-2e681db03377 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.757043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d720e-66f5-43a0-94c7-2bb1ed1b73a6 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.768275 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 781/1590] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:32.771561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 810/1591] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:32.771561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d720e-66f5-43a0-94c7-2bb1ed1b73a6 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.772624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6b97c-1a29-4011-aa56-5a65d2c70cf1 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.772918 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6b97c-1a29-4011-aa56-5a65d2c70cf1 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.773192 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6b97c-1a29-4011-aa56-5a65d2c70cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.780421 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d720e-66f5-43a0-94c7-2bb1ed1b73a6 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.785953 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da5d35-bba9-40e8-81cf-282bfb4b39c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.786536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da5d35-bba9-40e8-81cf-282bfb4b39c5 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.786822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da5d35-bba9-40e8-81cf-282bfb4b39c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.787128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da5d35-bba9-40e8-81cf-282bfb4b39c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.787833 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da5d35-bba9-40e8-81cf-282bfb4b39c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.789408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da5d35-bba9-40e8-81cf-282bfb4b39c5 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.799270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 811/1592] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:32.800032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da5d35-bba9-40e8-81cf-282bfb4b39c5 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.803998 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-5ce6b97c-1a29-4011-aa56-5a65d2c70cf1 None None] MFA Rules not processed for user `a9eddd862d4443e796684bbefd8264fb`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:32.805206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-5ce6b97c-1a29-4011-aa56-5a65d2c70cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:32.810556 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da5d35-bba9-40e8-81cf-282bfb4b39c5 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.817752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec9dc8-ba4a-45db-ae41-c25f413e2f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.818776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec9dc8-ba4a-45db-ae41-c25f413e2f98 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.819064 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec9dc8-ba4a-45db-ae41-c25f413e2f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.819367 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec9dc8-ba4a-45db-ae41-c25f413e2f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.820259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec9dc8-ba4a-45db-ae41-c25f413e2f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.822151 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec9dc8-ba4a-45db-ae41-c25f413e2f98 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.839823 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 812/1593] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:32.839823 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec9dc8-ba4a-45db-ae41-c25f413e2f98 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.850902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec9dc8-ba4a-45db-ae41-c25f413e2f98 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.855931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cce745-6a70-4f4f-920c-40207bfe81e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.856586 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cce745-6a70-4f4f-920c-40207bfe81e3 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.856865 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cce745-6a70-4f4f-920c-40207bfe81e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.857132 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cce745-6a70-4f4f-920c-40207bfe81e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.857871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cce745-6a70-4f4f-920c-40207bfe81e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.860184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cce745-6a70-4f4f-920c-40207bfe81e3 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.870269 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 813/1594] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:32.871093 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cce745-6a70-4f4f-920c-40207bfe81e3 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.882185 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cce745-6a70-4f4f-920c-40207bfe81e3 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.883350 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 782/1595] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:32.884041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6b97c-1a29-4011-aa56-5a65d2c70cf1 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.890809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac97220-fa3b-41f8-8c67-acc30e841771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.891628 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac97220-fa3b-41f8-8c67-acc30e841771 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.891832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac97220-fa3b-41f8-8c67-acc30e841771 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.892110 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac97220-fa3b-41f8-8c67-acc30e841771 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.892861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac97220-fa3b-41f8-8c67-acc30e841771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.894451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac97220-fa3b-41f8-8c67-acc30e841771 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.894738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6b97c-1a29-4011-aa56-5a65d2c70cf1 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.900868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4ddb2-b34e-4ded-9cea-ec96e3cd9b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.901622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4ddb2-b34e-4ded-9cea-ec96e3cd9b92 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.901943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4ddb2-b34e-4ded-9cea-ec96e3cd9b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.902216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4ddb2-b34e-4ded-9cea-ec96e3cd9b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.902940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4ddb2-b34e-4ded-9cea-ec96e3cd9b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.904565 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4ddb2-b34e-4ded-9cea-ec96e3cd9b92 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.908206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 814/1596] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:32.908206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac97220-fa3b-41f8-8c67-acc30e841771 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.915796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 783/1597] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:32.916433 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4ddb2-b34e-4ded-9cea-ec96e3cd9b92 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.924753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac97220-fa3b-41f8-8c67-acc30e841771 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.926677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4ddb2-b34e-4ded-9cea-ec96e3cd9b92 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.930434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06752556-58bf-4352-a192-37baafeacff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.931549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06752556-58bf-4352-a192-37baafeacff0 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.931687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fbc53-9464-4715-b120-07e1e37d2304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:32.931798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06752556-58bf-4352-a192-37baafeacff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.932050 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06752556-58bf-4352-a192-37baafeacff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.933375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06752556-58bf-4352-a192-37baafeacff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.933521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fbc53-9464-4715-b120-07e1e37d2304 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:32.933521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fbc53-9464-4715-b120-07e1e37d2304 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:32.933813 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fbc53-9464-4715-b120-07e1e37d2304 admin admin] PATH_INFO: `/v3/projects/b9ed303802f64743b7c3f2790237fca4/users/b2504afcef924cd99b672175f4adcbcd/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:32.934910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06752556-58bf-4352-a192-37baafeacff0 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.947551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 815/1598] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:32.949304 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fbc53-9464-4715-b120-07e1e37d2304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9ed303802f64743b7c3f2790237fca4, user_id=b2504afcef924cd99b672175f4adcbcd, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:32.951911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06752556-58bf-4352-a192-37baafeacff0 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:32.952301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fbc53-9464-4715-b120-07e1e37d2304 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:32.963307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06752556-58bf-4352-a192-37baafeacff0 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:32.974398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 784/1599] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:32 2022] PUT /identity/v3/projects/b9ed303802f64743b7c3f2790237fca4/users/b2504afcef924cd99b672175f4adcbcd/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:32.975561 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fbc53-9464-4715-b120-07e1e37d2304 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:33.027617 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a23394-f694-4297-97aa-3651b7d849cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:33.028210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a23394-f694-4297-97aa-3651b7d849cb admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.028496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a23394-f694-4297-97aa-3651b7d849cb admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.028771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a23394-f694-4297-97aa-3651b7d849cb admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.029491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a23394-f694-4297-97aa-3651b7d849cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:33.031637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a23394-f694-4297-97aa-3651b7d849cb admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:33.045975 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 816/1600] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:33.047615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a23394-f694-4297-97aa-3651b7d849cb admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:33.054437 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fbc53-9464-4715-b120-07e1e37d2304 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:33.059762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361c5db-db4f-4286-8a9b-e6b113846a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:33.060500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361c5db-db4f-4286-8a9b-e6b113846a53 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.060784 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361c5db-db4f-4286-8a9b-e6b113846a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.061066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361c5db-db4f-4286-8a9b-e6b113846a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.061789 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361c5db-db4f-4286-8a9b-e6b113846a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:33.063551 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361c5db-db4f-4286-8a9b-e6b113846a53 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:33.071906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a23394-f694-4297-97aa-3651b7d849cb admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:33.075559 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 785/1601] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:33.076643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361c5db-db4f-4286-8a9b-e6b113846a53 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:33.078405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c12bf-4067-48c6-95a6-f3eb769cdb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:33.079225 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c12bf-4067-48c6-95a6-f3eb769cdb66 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.080817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c12bf-4067-48c6-95a6-f3eb769cdb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.081118 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c12bf-4067-48c6-95a6-f3eb769cdb66 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.081956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c12bf-4067-48c6-95a6-f3eb769cdb66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:33.084412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c12bf-4067-48c6-95a6-f3eb769cdb66 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:33.090015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361c5db-db4f-4286-8a9b-e6b113846a53 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:33.096180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a09b58-3cf3-4005-853b-c374b8a864d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:33.096636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a09b58-3cf3-4005-853b-c374b8a864d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.096930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a09b58-3cf3-4005-853b-c374b8a864d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.097191 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a09b58-3cf3-4005-853b-c374b8a864d2 admin admin] PATH_INFO: `/v3/projects/31125b05c5e54ea28bac344b98366385/users/a2228dffa3514c39bee2f7c2f41fbc60/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.118188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a09b58-3cf3-4005-853b-c374b8a864d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31125b05c5e54ea28bac344b98366385, user_id=a2228dffa3514c39bee2f7c2f41fbc60, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:33.121839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a09b58-3cf3-4005-853b-c374b8a864d2 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:33.125371 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 817/1602] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:33 2022] POST /identity/v3/projects => generated 371 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:33.126249 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c12bf-4067-48c6-95a6-f3eb769cdb66 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:33.146472 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 786/1603] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:33 2022] PUT /identity/v3/projects/31125b05c5e54ea28bac344b98366385/users/a2228dffa3514c39bee2f7c2f41fbc60/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:33.147559 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a09b58-3cf3-4005-853b-c374b8a864d2 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:33.195090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c12bf-4067-48c6-95a6-f3eb769cdb66 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:33.219982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a09b58-3cf3-4005-853b-c374b8a864d2 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:33.226337 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b335891-f887-43d6-8790-b77d8c8231e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:33.227348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b335891-f887-43d6-8790-b77d8c8231e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.227725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b335891-f887-43d6-8790-b77d8c8231e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.228073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b335891-f887-43d6-8790-b77d8c8231e3 admin admin] PATH_INFO: `/v3/projects/3582c1a023c645b38b91a5fa01be7667/users/f99208187e4e4d2eb9f52107ae8f50f2/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.244213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b335891-f887-43d6-8790-b77d8c8231e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3582c1a023c645b38b91a5fa01be7667, user_id=f99208187e4e4d2eb9f52107ae8f50f2, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:33.246979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b335891-f887-43d6-8790-b77d8c8231e3 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:33.257095 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad6ca6-a1e6-472e-9694-04514308eda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:33.258024 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad6ca6-a1e6-472e-9694-04514308eda6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.258309 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad6ca6-a1e6-472e-9694-04514308eda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.258580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad6ca6-a1e6-472e-9694-04514308eda6 admin admin] PATH_INFO: `/v3/projects/3e0d940b1aea46aea0fa187448375f04/users/3312802ff729418f90e6f25e6faf0ee0/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.261411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 787/1604] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:33 2022] PUT /identity/v3/projects/3582c1a023c645b38b91a5fa01be7667/users/f99208187e4e4d2eb9f52107ae8f50f2/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:33.262608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed252f-1698-464f-b137-032bd5f33656 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.262906 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed252f-1698-464f-b137-032bd5f33656 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.263199 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed252f-1698-464f-b137-032bd5f33656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.276701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad6ca6-a1e6-472e-9694-04514308eda6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0d940b1aea46aea0fa187448375f04, user_id=3312802ff729418f90e6f25e6faf0ee0, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:33.279428 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad6ca6-a1e6-472e-9694-04514308eda6 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:33.290413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-b0ed252f-1698-464f-b137-032bd5f33656 None None] MFA Rules not processed for user `b2504afcef924cd99b672175f4adcbcd`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:33.291637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b0ed252f-1698-464f-b137-032bd5f33656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:33.298252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 818/1605] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:33 2022] PUT /identity/v3/projects/3e0d940b1aea46aea0fa187448375f04/users/3312802ff729418f90e6f25e6faf0ee0/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:33.299226 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad6ca6-a1e6-472e-9694-04514308eda6 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:33.362908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad6ca6-a1e6-472e-9694-04514308eda6 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:33.367585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 788/1606] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:33.367585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed252f-1698-464f-b137-032bd5f33656 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:33.371794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0db93-7021-4970-9333-0fb891250364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:33.373132 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0db93-7021-4970-9333-0fb891250364 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.373258 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0db93-7021-4970-9333-0fb891250364 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.373517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0db93-7021-4970-9333-0fb891250364 admin admin] PATH_INFO: `/v3/projects/78039fb4f0bd454b8aea3db748343263/users/90fdc96d6e6649a498bb3a8659fcc030/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.377008 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed252f-1698-464f-b137-032bd5f33656 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:33.382254 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4ace5-5af8-4ba1-a0f6-7ac30ca0cb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:33.383271 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4ace5-5af8-4ba1-a0f6-7ac30ca0cb79 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.383582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4ace5-5af8-4ba1-a0f6-7ac30ca0cb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.383987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4ace5-5af8-4ba1-a0f6-7ac30ca0cb79 admin admin] PATH_INFO: `/v3/projects/74dc3d60f0bc4ceb85d32720240c4032/users/9e73b40a50c1493bbee6142044605826/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.388780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0db93-7021-4970-9333-0fb891250364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78039fb4f0bd454b8aea3db748343263, user_id=90fdc96d6e6649a498bb3a8659fcc030, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:33.392041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0db93-7021-4970-9333-0fb891250364 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:33.400828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4ace5-5af8-4ba1-a0f6-7ac30ca0cb79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74dc3d60f0bc4ceb85d32720240c4032, user_id=9e73b40a50c1493bbee6142044605826, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:33.403504 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4ace5-5af8-4ba1-a0f6-7ac30ca0cb79 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:33.415635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 819/1607] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:33 2022] PUT /identity/v3/projects/78039fb4f0bd454b8aea3db748343263/users/90fdc96d6e6649a498bb3a8659fcc030/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:33.417399 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0db93-7021-4970-9333-0fb891250364 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:33.424019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 789/1608] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:33 2022] PUT /identity/v3/projects/74dc3d60f0bc4ceb85d32720240c4032/users/9e73b40a50c1493bbee6142044605826/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:33.425333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5111160-6e5e-4a1b-a0da-a972a8f887c4 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.425622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5111160-6e5e-4a1b-a0da-a972a8f887c4 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.425901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5111160-6e5e-4a1b-a0da-a972a8f887c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.457176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-c5111160-6e5e-4a1b-a0da-a972a8f887c4 None None] MFA Rules not processed for user `a2228dffa3514c39bee2f7c2f41fbc60`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:33.458390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c5111160-6e5e-4a1b-a0da-a972a8f887c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:33.506585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0db93-7021-4970-9333-0fb891250364 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:33.537861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 790/1609] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:33.539191 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b0f42-0972-4a7e-ade7-096522ec1e46 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.539489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b0f42-0972-4a7e-ade7-096522ec1e46 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.539772 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b0f42-0972-4a7e-ade7-096522ec1e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.567090 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-3b2b0f42-0972-4a7e-ade7-096522ec1e46 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:33.570143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-3b2b0f42-0972-4a7e-ade7-096522ec1e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:33.587711 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f598e2f-5486-4f8e-86d0-e718a752d6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:33.588405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598e2f-5486-4f8e-86d0-e718a752d6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.588684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598e2f-5486-4f8e-86d0-e718a752d6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.589059 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598e2f-5486-4f8e-86d0-e718a752d6a5 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.589641 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f598e2f-5486-4f8e-86d0-e718a752d6a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:33.591586 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f598e2f-5486-4f8e-86d0-e718a752d6a5 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:33.618206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 820/1610] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:33 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:33.619736 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a0db5-1391-4531-8f3b-90baf068ab24 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.620077 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a0db5-1391-4531-8f3b-90baf068ab24 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.620367 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a0db5-1391-4531-8f3b-90baf068ab24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.644959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 791/1611] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:33.645695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b0f42-0972-4a7e-ade7-096522ec1e46 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:33.647693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-8b9a0db5-1391-4531-8f3b-90baf068ab24 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:33.649716 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-8b9a0db5-1391-4531-8f3b-90baf068ab24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:33.659161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 821/1612] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:33.660537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f4938-4c0c-417c-b0dd-a4fdf5ae90b4 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.660988 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f4938-4c0c-417c-b0dd-a4fdf5ae90b4 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.661279 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f4938-4c0c-417c-b0dd-a4fdf5ae90b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.672063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b0f42-0972-4a7e-ade7-096522ec1e46 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:33.680528 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2f8f4938-4c0c-417c-b0dd-a4fdf5ae90b4 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:33.681731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2f8f4938-4c0c-417c-b0dd-a4fdf5ae90b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:33.684495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72120579-0ee6-4552-894d-3750954fd48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:33.685301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72120579-0ee6-4552-894d-3750954fd48d admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.686198 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72120579-0ee6-4552-894d-3750954fd48d admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.686641 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72120579-0ee6-4552-894d-3750954fd48d admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.687955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72120579-0ee6-4552-894d-3750954fd48d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:33.689871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72120579-0ee6-4552-894d-3750954fd48d admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:33.692725 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 822/1613] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:33.694834 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5070ae-a1bd-408c-a8de-9b46dfe39233 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.695257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5070ae-a1bd-408c-a8de-9b46dfe39233 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.695743 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5070ae-a1bd-408c-a8de-9b46dfe39233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.728954 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-db5070ae-a1bd-408c-a8de-9b46dfe39233 None None] MFA Rules not processed for user `9e73b40a50c1493bbee6142044605826`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:33.730293 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-db5070ae-a1bd-408c-a8de-9b46dfe39233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:33.734450 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 792/1614] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:33 2022] POST /identity/v3/projects => generated 369 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:33.741583 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7c3b0-1dcc-4b25-a884-5fb8db4421d2 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.741891 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7c3b0-1dcc-4b25-a884-5fb8db4421d2 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.742885 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7c3b0-1dcc-4b25-a884-5fb8db4421d2 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.744937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 793/1615] 10.222.0.25 () {54 vars in 961 bytes} [Mon May 9 18:29:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:29:33.745701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7c3b0-1dcc-4b25-a884-5fb8db4421d2 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:33.824342 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 823/1616] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:33.825821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5070ae-a1bd-408c-a8de-9b46dfe39233 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:33.842936 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7c3b0-1dcc-4b25-a884-5fb8db4421d2 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:33.848511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34af62-d423-4098-947e-e0dcffeeda9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:33.849328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34af62-d423-4098-947e-e0dcffeeda9b admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.849680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34af62-d423-4098-947e-e0dcffeeda9b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.850029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34af62-d423-4098-947e-e0dcffeeda9b admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.850837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34af62-d423-4098-947e-e0dcffeeda9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:33.852591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34af62-d423-4098-947e-e0dcffeeda9b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:33.866731 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 794/1617] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:33.867764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34af62-d423-4098-947e-e0dcffeeda9b admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:33.910186 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5070ae-a1bd-408c-a8de-9b46dfe39233 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:33.916196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fc610-dd5e-4c77-9b88-a7f676b22722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:33.916962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fc610-dd5e-4c77-9b88-a7f676b22722 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.917311 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fc610-dd5e-4c77-9b88-a7f676b22722 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.917659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fc610-dd5e-4c77-9b88-a7f676b22722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.927896 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3fc610-dd5e-4c77-9b88-a7f676b22722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:33.929555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3fc610-dd5e-4c77-9b88-a7f676b22722 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:33.941615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 824/1618] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:33.942615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fc610-dd5e-4c77-9b88-a7f676b22722 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:33.951005 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34af62-d423-4098-947e-e0dcffeeda9b admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:33.956405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fc610-dd5e-4c77-9b88-a7f676b22722 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:33.961117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad23407-17f5-4f6e-b677-a9b7b8fa3dc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:33.961912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad23407-17f5-4f6e-b677-a9b7b8fa3dc3 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.962258 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad23407-17f5-4f6e-b677-a9b7b8fa3dc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.962604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad23407-17f5-4f6e-b677-a9b7b8fa3dc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.964008 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674ac2a-83c4-4a7a-8c8e-760cdfa499fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:33.965098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674ac2a-83c4-4a7a-8c8e-760cdfa499fa service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:33.965535 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674ac2a-83c4-4a7a-8c8e-760cdfa499fa service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:33.969349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674ac2a-83c4-4a7a-8c8e-760cdfa499fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:33.974050 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad23407-17f5-4f6e-b677-a9b7b8fa3dc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:33.975965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad23407-17f5-4f6e-b677-a9b7b8fa3dc3 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:33.985780 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2674ac2a-83c4-4a7a-8c8e-760cdfa499fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:33.987794 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2674ac2a-83c4-4a7a-8c8e-760cdfa499fa service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:33.990502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 825/1619] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:33.991300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad23407-17f5-4f6e-b677-a9b7b8fa3dc3 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:34.006944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 795/1620] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:34.008645 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ed7d4-4c6b-4e07-93de-3ef049d15128 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.008802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad23407-17f5-4f6e-b677-a9b7b8fa3dc3 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:34.009227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ed7d4-4c6b-4e07-93de-3ef049d15128 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.009907 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ed7d4-4c6b-4e07-93de-3ef049d15128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.022356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f9390-c5f8-42d7-b298-de64959f7812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:34.023034 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f9390-c5f8-42d7-b298-de64959f7812 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.023379 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f9390-c5f8-42d7-b298-de64959f7812 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.024392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f9390-c5f8-42d7-b298-de64959f7812 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.024392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f9390-c5f8-42d7-b298-de64959f7812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:34.028228 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f9390-c5f8-42d7-b298-de64959f7812 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:34.052907 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-011ed7d4-4c6b-4e07-93de-3ef049d15128 None None] MFA Rules not processed for user `f769ee9f31354e5296645fb1e3bd84b6`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:34.056262 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-011ed7d4-4c6b-4e07-93de-3ef049d15128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:34.070603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 826/1621] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:33 2022] POST /identity/v3/users => generated 291 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:34.070603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d387774-a3a6-479a-9bce-e039c61bbbdf None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.070603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d387774-a3a6-479a-9bce-e039c61bbbdf None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.070972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d387774-a3a6-479a-9bce-e039c61bbbdf None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.078325 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 827/1622] 10.222.0.25 () {54 vars in 961 bytes} [Mon May 9 18:29:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:29:34.079519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d387774-a3a6-479a-9bce-e039c61bbbdf None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:34.103235 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d387774-a3a6-479a-9bce-e039c61bbbdf None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:34.110751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef37f703-1cb9-40b3-9564-0a1c967b4c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:34.111914 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37f703-1cb9-40b3-9564-0a1c967b4c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.112527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37f703-1cb9-40b3-9564-0a1c967b4c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.112954 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37f703-1cb9-40b3-9564-0a1c967b4c80 admin admin] PATH_INFO: `/v3/projects/e05dfefe5f6749e38a51fda868ceb829/users/a328ccee066940be9832ffe20e61d741/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.138260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef37f703-1cb9-40b3-9564-0a1c967b4c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e05dfefe5f6749e38a51fda868ceb829, user_id=a328ccee066940be9832ffe20e61d741, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:34.142356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef37f703-1cb9-40b3-9564-0a1c967b4c80 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:34.162278 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 828/1623] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:34 2022] PUT /identity/v3/projects/e05dfefe5f6749e38a51fda868ceb829/users/a328ccee066940be9832ffe20e61d741/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:34.163684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef37f703-1cb9-40b3-9564-0a1c967b4c80 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:34.207217 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 796/1624] 10.222.0.25 () {58 vars in 1058 bytes} [Mon May 9 18:29:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:34.210438 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e9328-4cb2-4862-8f60-5d9c1425b772 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.210739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e9328-4cb2-4862-8f60-5d9c1425b772 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.211025 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e9328-4cb2-4862-8f60-5d9c1425b772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.259049 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-187e9328-4cb2-4862-8f60-5d9c1425b772 None None] MFA Rules not processed for user `f769ee9f31354e5296645fb1e3bd84b6`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:34.262330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-187e9328-4cb2-4862-8f60-5d9c1425b772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:34.278680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef37f703-1cb9-40b3-9564-0a1c967b4c80 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:34.301151 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d7690-49b2-465f-b10b-e6cc823f21ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:34.302231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d7690-49b2-465f-b10b-e6cc823f21ac admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.302728 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d7690-49b2-465f-b10b-e6cc823f21ac admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.303214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d7690-49b2-465f-b10b-e6cc823f21ac admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.304146 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d7690-49b2-465f-b10b-e6cc823f21ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:34.306408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d7690-49b2-465f-b10b-e6cc823f21ac admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:34.321335 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 829/1625] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:34.322469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d7690-49b2-465f-b10b-e6cc823f21ac admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:34.339704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d7690-49b2-465f-b10b-e6cc823f21ac admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:34.356471 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86e32b-6af7-4c05-9986-70383d4ac58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:34.357157 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86e32b-6af7-4c05-9986-70383d4ac58f admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.357437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86e32b-6af7-4c05-9986-70383d4ac58f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.357708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86e32b-6af7-4c05-9986-70383d4ac58f admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.358429 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86e32b-6af7-4c05-9986-70383d4ac58f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:34.360576 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86e32b-6af7-4c05-9986-70383d4ac58f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:34.388236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 830/1626] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:34.389143 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86e32b-6af7-4c05-9986-70383d4ac58f admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:34.396595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 797/1627] 10.222.0.25 () {58 vars in 1058 bytes} [Mon May 9 18:29:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:34.405010 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e9328-4cb2-4862-8f60-5d9c1425b772 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:34.421766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e9328-4cb2-4862-8f60-5d9c1425b772 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:34.426948 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ef27c-f3db-4837-97c7-18c46a8ab671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:34.428296 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ef27c-f3db-4837-97c7-18c46a8ab671 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.428604 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ef27c-f3db-4837-97c7-18c46a8ab671 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.429065 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ef27c-f3db-4837-97c7-18c46a8ab671 admin admin] PATH_INFO: `/v3/projects/7487ca4e595a4b058f99731ff57cb1cf/users/3620df13ffeb426497723934c8a26626/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.435286 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86e32b-6af7-4c05-9986-70383d4ac58f admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:34.447356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b13952-5015-455b-9986-28a53382ce47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:34.448698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b13952-5015-455b-9986-28a53382ce47 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.448964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ef27c-f3db-4837-97c7-18c46a8ab671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7487ca4e595a4b058f99731ff57cb1cf, user_id=3620df13ffeb426497723934c8a26626, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:34.449172 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b13952-5015-455b-9986-28a53382ce47 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.452013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ef27c-f3db-4837-97c7-18c46a8ab671 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:34.456184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b13952-5015-455b-9986-28a53382ce47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.473833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 798/1628] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:34 2022] PUT /identity/v3/projects/7487ca4e595a4b058f99731ff57cb1cf/users/3620df13ffeb426497723934c8a26626/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:34.474873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ef27c-f3db-4837-97c7-18c46a8ab671 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:34.548626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b13952-5015-455b-9986-28a53382ce47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:34.550434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b13952-5015-455b-9986-28a53382ce47 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:34.560088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ef27c-f3db-4837-97c7-18c46a8ab671 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:34.566026 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 831/1629] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:34.567257 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba729f8-db1f-4bd3-bcc0-349ab71a33e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:34.567575 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b13952-5015-455b-9986-28a53382ce47 service nova] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:34.568859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba729f8-db1f-4bd3-bcc0-349ab71a33e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.569327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba729f8-db1f-4bd3-bcc0-349ab71a33e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.569792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba729f8-db1f-4bd3-bcc0-349ab71a33e0 admin admin] PATH_INFO: `/v3/projects/e05dfefe5f6749e38a51fda868ceb829/users/a328ccee066940be9832ffe20e61d741/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.572934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba729f8-db1f-4bd3-bcc0-349ab71a33e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e05dfefe5f6749e38a51fda868ceb829, user_id=a328ccee066940be9832ffe20e61d741, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:34.576423 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba729f8-db1f-4bd3-bcc0-349ab71a33e0 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:34.580472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b13952-5015-455b-9986-28a53382ce47 service nova] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:34.591063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 799/1630] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:34 2022] PUT /identity/v3/projects/e05dfefe5f6749e38a51fda868ceb829/users/a328ccee066940be9832ffe20e61d741/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:34.592488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba729f8-db1f-4bd3-bcc0-349ab71a33e0 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:34.714039 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d181b40-b29b-453d-a7fc-fb59d182d930 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:34.715111 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d181b40-b29b-453d-a7fc-fb59d182d930 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.715640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d181b40-b29b-453d-a7fc-fb59d182d930 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.717228 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d181b40-b29b-453d-a7fc-fb59d182d930 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.753133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba729f8-db1f-4bd3-bcc0-349ab71a33e0 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:34.765210 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb9aaa-4308-4a37-b9f8-0bdd0a0d70ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:34.767098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb9aaa-4308-4a37-b9f8-0bdd0a0d70ad admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.767459 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb9aaa-4308-4a37-b9f8-0bdd0a0d70ad admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.767744 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb9aaa-4308-4a37-b9f8-0bdd0a0d70ad admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.770550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb9aaa-4308-4a37-b9f8-0bdd0a0d70ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:34.773607 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb9aaa-4308-4a37-b9f8-0bdd0a0d70ad admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:34.802650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d181b40-b29b-453d-a7fc-fb59d182d930 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:34.804111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 800/1631] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:34.807967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d181b40-b29b-453d-a7fc-fb59d182d930 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:34.818454 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea867cf-9516-4fee-a011-ab614786130a None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.818755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea867cf-9516-4fee-a011-ab614786130a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.819227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea867cf-9516-4fee-a011-ab614786130a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.841717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 832/1632] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:34.842776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d181b40-b29b-453d-a7fc-fb59d182d930 service nova] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:34.867946 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d181b40-b29b-453d-a7fc-fb59d182d930 service nova] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:34.877970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd16413-7fa5-4367-b996-daad215b2026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:34.879343 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd16413-7fa5-4367-b996-daad215b2026 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.879683 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd16413-7fa5-4367-b996-daad215b2026 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.880068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd16413-7fa5-4367-b996-daad215b2026 admin admin] PATH_INFO: `/v3/projects/7487ca4e595a4b058f99731ff57cb1cf/users/3620df13ffeb426497723934c8a26626/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.886162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd16413-7fa5-4367-b996-daad215b2026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7487ca4e595a4b058f99731ff57cb1cf, user_id=3620df13ffeb426497723934c8a26626, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:34.891521 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd16413-7fa5-4367-b996-daad215b2026 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:34.900340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-3ea867cf-9516-4fee-a011-ab614786130a None None] MFA Rules not processed for user `a328ccee066940be9832ffe20e61d741`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:34.901726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-3ea867cf-9516-4fee-a011-ab614786130a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:34.930926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 833/1633] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:34 2022] PUT /identity/v3/projects/7487ca4e595a4b058f99731ff57cb1cf/users/3620df13ffeb426497723934c8a26626/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:34.932604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951d267-6461-4a6a-9fe9-8cf03177767a None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:34.933045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951d267-6461-4a6a-9fe9-8cf03177767a None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:34.933405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951d267-6461-4a6a-9fe9-8cf03177767a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:34.992792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-0951d267-6461-4a6a-9fe9-8cf03177767a None None] MFA Rules not processed for user `3620df13ffeb426497723934c8a26626`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:34.994162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-0951d267-6461-4a6a-9fe9-8cf03177767a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:35.077205 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 801/1634] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:34 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:35.088100 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea867cf-9516-4fee-a011-ab614786130a None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:35.113128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 834/1635] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:34 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:35.140779 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951d267-6461-4a6a-9fe9-8cf03177767a None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:35.236744 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea867cf-9516-4fee-a011-ab614786130a None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:35.300144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951d267-6461-4a6a-9fe9-8cf03177767a None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:35.304481 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0210a-d54e-46bf-8f70-efab65777a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:35.305197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0210a-d54e-46bf-8f70-efab65777a00 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:35.305532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0210a-d54e-46bf-8f70-efab65777a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:35.306002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0210a-d54e-46bf-8f70-efab65777a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:35.323205 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0210a-d54e-46bf-8f70-efab65777a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:35.328307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0210a-d54e-46bf-8f70-efab65777a00 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:35.352730 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d4d1d-673b-4619-b1b9-33a896efcefd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:35.354622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d4d1d-673b-4619-b1b9-33a896efcefd service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:35.355276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d4d1d-673b-4619-b1b9-33a896efcefd service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:35.355809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d4d1d-673b-4619-b1b9-33a896efcefd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:35.366451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 835/1636] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:35 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:35.368788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54438744-7ff9-43bd-a4fb-a216357922b3 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:35.368788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54438744-7ff9-43bd-a4fb-a216357922b3 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:35.369122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54438744-7ff9-43bd-a4fb-a216357922b3 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:35.381488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 836/1637] 10.222.0.25 () {54 vars in 912 bytes} [Mon May 9 18:29:35 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:29:35.385989 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6fbb4-692b-41e1-bd29-10e64e53ba7b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:35.386282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6fbb4-692b-41e1-bd29-10e64e53ba7b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:35.386561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6fbb4-692b-41e1-bd29-10e64e53ba7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:35.465260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-7dd6fbb4-692b-41e1-bd29-10e64e53ba7b None None] MFA Rules not processed for user `3312802ff729418f90e6f25e6faf0ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:35.466494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7dd6fbb4-692b-41e1-bd29-10e64e53ba7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:35.501729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d4d1d-673b-4619-b1b9-33a896efcefd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:35.507145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d4d1d-673b-4619-b1b9-33a896efcefd service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:35.547098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 802/1638] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:35 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:29:35.548807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9c3cf-6a2e-4a0a-bb85-aa9c9ad10171 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:35.550512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9c3cf-6a2e-4a0a-bb85-aa9c9ad10171 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:35.550903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9c3cf-6a2e-4a0a-bb85-aa9c9ad10171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:35.554071 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 837/1639] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:35.557435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a93df3-8e21-46cc-b969-d1b1dc73c8f2 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:35.557729 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a93df3-8e21-46cc-b969-d1b1dc73c8f2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:35.558011 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a93df3-8e21-46cc-b969-d1b1dc73c8f2 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:35.562589 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 838/1640] 10.222.0.25 () {54 vars in 912 bytes} [Mon May 9 18:29:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:29:35.566350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fd976-0dfd-40d2-a4ac-1b682562bb74 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:35.566665 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fd976-0dfd-40d2-a4ac-1b682562bb74 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:35.566923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fd976-0dfd-40d2-a4ac-1b682562bb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:35.595879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-66e9c3cf-6a2e-4a0a-bb85-aa9c9ad10171 None None] MFA Rules not processed for user `f769ee9f31354e5296645fb1e3bd84b6`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:35.598544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-66e9c3cf-6a2e-4a0a-bb85-aa9c9ad10171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:35.612080 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-e68fd976-0dfd-40d2-a4ac-1b682562bb74 None None] MFA Rules not processed for user `f769ee9f31354e5296645fb1e3bd84b6`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:35.612080 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e68fd976-0dfd-40d2-a4ac-1b682562bb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:35.836590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 839/1641] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:29:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:35.842169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cdbf4-c4c6-42b0-853a-f2eefed9b8e1 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:35.842487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cdbf4-c4c6-42b0-853a-f2eefed9b8e1 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:35.842770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cdbf4-c4c6-42b0-853a-f2eefed9b8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:35.864204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 803/1642] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:29:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:35.867386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9c3cf-6a2e-4a0a-bb85-aa9c9ad10171 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:35.887157 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-5f5cdbf4-c4c6-42b0-853a-f2eefed9b8e1 None None] MFA Rules not processed for user `3312802ff729418f90e6f25e6faf0ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:35.888389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-5f5cdbf4-c4c6-42b0-853a-f2eefed9b8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:35.890950 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9c3cf-6a2e-4a0a-bb85-aa9c9ad10171 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:35.897580 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0bf97-caf3-4e6a-a6e8-ea5548057e03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:35.898217 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0bf97-caf3-4e6a-a6e8-ea5548057e03 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:35.898555 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0bf97-caf3-4e6a-a6e8-ea5548057e03 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:35.898769 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0bf97-caf3-4e6a-a6e8-ea5548057e03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:35.904223 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 840/1643] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:35.917606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0bf97-caf3-4e6a-a6e8-ea5548057e03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:35.919757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0bf97-caf3-4e6a-a6e8-ea5548057e03 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:35.948471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 804/1644] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:36.211791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cdbf4-c4c6-42b0-853a-f2eefed9b8e1 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:36.227029 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cdbf4-c4c6-42b0-853a-f2eefed9b8e1 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:36.232286 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f291862-1b0b-4116-b196-38cec99c34bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:36.233463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f291862-1b0b-4116-b196-38cec99c34bf service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:36.233970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f291862-1b0b-4116-b196-38cec99c34bf service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:36.234403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f291862-1b0b-4116-b196-38cec99c34bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:36.250018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f291862-1b0b-4116-b196-38cec99c34bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:36.252261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f291862-1b0b-4116-b196-38cec99c34bf service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:36.274413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 841/1645] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:36.366480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0bf97-caf3-4e6a-a6e8-ea5548057e03 service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:36.386803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0bf97-caf3-4e6a-a6e8-ea5548057e03 service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:36.407563 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e5507-98b9-4eb2-b02b-b0c88d7d1633 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:36.410618 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e5507-98b9-4eb2-b02b-b0c88d7d1633 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:36.411163 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e5507-98b9-4eb2-b02b-b0c88d7d1633 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:36.411701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e5507-98b9-4eb2-b02b-b0c88d7d1633 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:36.427597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e5507-98b9-4eb2-b02b-b0c88d7d1633 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:36.434542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e5507-98b9-4eb2-b02b-b0c88d7d1633 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:36.464891 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 805/1646] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:37.232434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f291862-1b0b-4116-b196-38cec99c34bf service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:37.368528 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e5507-98b9-4eb2-b02b-b0c88d7d1633 service nova] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:37.398804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f291862-1b0b-4116-b196-38cec99c34bf service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:37.420951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25570a-2068-4e79-bff8-ab55ea0e5bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:37.422166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25570a-2068-4e79-bff8-ab55ea0e5bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:37.422701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25570a-2068-4e79-bff8-ab55ea0e5bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:37.423707 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25570a-2068-4e79-bff8-ab55ea0e5bd5 admin admin] PATH_INFO: `/v3/users/a9eddd862d4443e796684bbefd8264fb` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:37.425665 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25570a-2068-4e79-bff8-ab55ea0e5bd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9eddd862d4443e796684bbefd8264fb)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:37.430163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25570a-2068-4e79-bff8-ab55ea0e5bd5 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:37.489264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e5507-98b9-4eb2-b02b-b0c88d7d1633 service nova] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:37.497737 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adac865-37cc-4c0c-aff6-73d00881015d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:37.498977 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adac865-37cc-4c0c-aff6-73d00881015d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:37.500405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adac865-37cc-4c0c-aff6-73d00881015d admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:37.500891 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adac865-37cc-4c0c-aff6-73d00881015d admin admin] PATH_INFO: `/v3/users/a2228dffa3514c39bee2f7c2f41fbc60` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:37.503014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adac865-37cc-4c0c-aff6-73d00881015d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2228dffa3514c39bee2f7c2f41fbc60)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:37.504732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: WARNING py.warnings [None req-4adac865-37cc-4c0c-aff6-73d00881015d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 18:29:37.504732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: warnings.warn(msg) May 09 18:29:37.504732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: May 09 18:29:37.506757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adac865-37cc-4c0c-aff6-73d00881015d admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:37.536568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-4c25570a-2068-4e79-bff8-ab55ea0e5bd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9eddd862d4443e796684bbefd8264fb'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:37.537513 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-4c25570a-2068-4e79-bff8-ab55ea0e5bd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9eddd862d4443e796684bbefd8264fb'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:37.551150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-4c25570a-2068-4e79-bff8-ab55ea0e5bd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9eddd862d4443e796684bbefd8264fb'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:37.579066 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-4c25570a-2068-4e79-bff8-ab55ea0e5bd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9eddd862d4443e796684bbefd8264fb'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:37.580341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-4c25570a-2068-4e79-bff8-ab55ea0e5bd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9eddd862d4443e796684bbefd8264fb'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:37.631914 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 842/1647] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:29:37 2022] DELETE /identity/v3/users/a9eddd862d4443e796684bbefd8264fb => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:37.634574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8597c5e-d2ca-4e75-b8c3-39bbbf4f1e00 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:37.635104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8597c5e-d2ca-4e75-b8c3-39bbbf4f1e00 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:37.636787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8597c5e-d2ca-4e75-b8c3-39bbbf4f1e00 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:37.643720 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 843/1648] 10.222.0.25 () {54 vars in 912 bytes} [Mon May 9 18:29:37 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:29:37.647031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da075866-0187-4d60-b5eb-719d8f90c726 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:37.647695 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da075866-0187-4d60-b5eb-719d8f90c726 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:37.648593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da075866-0187-4d60-b5eb-719d8f90c726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:37.664852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4adac865-37cc-4c0c-aff6-73d00881015d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2228dffa3514c39bee2f7c2f41fbc60'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:37.717498 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-da075866-0187-4d60-b5eb-719d8f90c726 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:37.719918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-da075866-0187-4d60-b5eb-719d8f90c726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:37.723267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4adac865-37cc-4c0c-aff6-73d00881015d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2228dffa3514c39bee2f7c2f41fbc60'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:37.725342 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4adac865-37cc-4c0c-aff6-73d00881015d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2228dffa3514c39bee2f7c2f41fbc60'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:37.752747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4adac865-37cc-4c0c-aff6-73d00881015d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2228dffa3514c39bee2f7c2f41fbc60'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:37.753954 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4adac865-37cc-4c0c-aff6-73d00881015d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2228dffa3514c39bee2f7c2f41fbc60'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:37.875884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 806/1649] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:29:37 2022] DELETE /identity/v3/users/a2228dffa3514c39bee2f7c2f41fbc60 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:29:37.881849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c805b1a-455f-4bee-9260-bca899bddf30 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:37.883821 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c805b1a-455f-4bee-9260-bca899bddf30 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:37.886013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c805b1a-455f-4bee-9260-bca899bddf30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:37.897100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 844/1650] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:37.901309 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cdbf2-2dc9-4702-808a-92f64ffa03eb None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:37.902571 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cdbf2-2dc9-4702-808a-92f64ffa03eb None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:37.902996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cdbf2-2dc9-4702-808a-92f64ffa03eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:37.950996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-f31cdbf2-2dc9-4702-808a-92f64ffa03eb None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:37.954118 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f31cdbf2-2dc9-4702-808a-92f64ffa03eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:38.009321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-1c805b1a-455f-4bee-9260-bca899bddf30 None None] MFA Rules not processed for user `6b2b6318776b422d9701351ba6b4ee27`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:38.013408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1c805b1a-455f-4bee-9260-bca899bddf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:38.117794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 845/1651] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:38.119010 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31cdbf2-2dc9-4702-808a-92f64ffa03eb None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:38.126942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f31cdbf2-2dc9-4702-808a-92f64ffa03eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:38.147886 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 807/1652] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:29:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:38.170952 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c805b1a-455f-4bee-9260-bca899bddf30 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:38.175329 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1c805b1a-455f-4bee-9260-bca899bddf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:38.221738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c805b1a-455f-4bee-9260-bca899bddf30 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:38.227939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789175ee-6130-4148-9fad-546343ea7f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:38.229082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789175ee-6130-4148-9fad-546343ea7f26 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:38.229499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789175ee-6130-4148-9fad-546343ea7f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:38.229782 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789175ee-6130-4148-9fad-546343ea7f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:38.243423 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789175ee-6130-4148-9fad-546343ea7f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:38.244946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789175ee-6130-4148-9fad-546343ea7f26 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:38.253605 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31cdbf2-2dc9-4702-808a-92f64ffa03eb None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:38.263787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c261a6f-4e4e-44de-8cf7-2bc9e4d17ce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:38.264463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c261a6f-4e4e-44de-8cf7-2bc9e4d17ce1 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:38.264749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c261a6f-4e4e-44de-8cf7-2bc9e4d17ce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:38.265070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c261a6f-4e4e-44de-8cf7-2bc9e4d17ce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:38.266723 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 808/1653] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:38.267623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789175ee-6130-4148-9fad-546343ea7f26 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:38.281471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789175ee-6130-4148-9fad-546343ea7f26 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:38.289878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c261a6f-4e4e-44de-8cf7-2bc9e4d17ce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:38.291487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c261a6f-4e4e-44de-8cf7-2bc9e4d17ce1 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:38.303130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f9a03-3e06-4528-9e8b-32aac4efb403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:38.303833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f9a03-3e06-4528-9e8b-32aac4efb403 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:38.304136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f9a03-3e06-4528-9e8b-32aac4efb403 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:38.304409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f9a03-3e06-4528-9e8b-32aac4efb403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:38.312953 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 846/1654] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:38.332926 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f9a03-3e06-4528-9e8b-32aac4efb403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:38.335574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f9a03-3e06-4528-9e8b-32aac4efb403 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:38.353036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 809/1655] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:38.889774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c261a6f-4e4e-44de-8cf7-2bc9e4d17ce1 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:38.893438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2c261a6f-4e4e-44de-8cf7-2bc9e4d17ce1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:38.915043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c261a6f-4e4e-44de-8cf7-2bc9e4d17ce1 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:38.940338 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbd1f1-1335-473a-8838-9c1abb879733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:38.941604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbd1f1-1335-473a-8838-9c1abb879733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:38.942121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbd1f1-1335-473a-8838-9c1abb879733 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:38.942588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbd1f1-1335-473a-8838-9c1abb879733 admin admin] PATH_INFO: `/v3/users/b2504afcef924cd99b672175f4adcbcd` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:38.947696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbd1f1-1335-473a-8838-9c1abb879733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2504afcef924cd99b672175f4adcbcd)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:38.949859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbd1f1-1335-473a-8838-9c1abb879733 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:39.060166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-33cbd1f1-1335-473a-8838-9c1abb879733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2504afcef924cd99b672175f4adcbcd'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:39.061465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-33cbd1f1-1335-473a-8838-9c1abb879733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2504afcef924cd99b672175f4adcbcd'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:39.080542 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-33cbd1f1-1335-473a-8838-9c1abb879733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2504afcef924cd99b672175f4adcbcd'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:39.088487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-33cbd1f1-1335-473a-8838-9c1abb879733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2504afcef924cd99b672175f4adcbcd'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:39.089184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-33cbd1f1-1335-473a-8838-9c1abb879733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2504afcef924cd99b672175f4adcbcd'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:39.114122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 847/1656] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:29:38 2022] DELETE /identity/v3/users/b2504afcef924cd99b672175f4adcbcd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:39.119631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef74771-dc5b-4fae-acc4-07d34796793e None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:39.120156 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef74771-dc5b-4fae-acc4-07d34796793e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:39.120934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef74771-dc5b-4fae-acc4-07d34796793e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:39.149065 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-5ef74771-dc5b-4fae-acc4-07d34796793e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:39.150500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-5ef74771-dc5b-4fae-acc4-07d34796793e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:39.310655 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 810/1657] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:39.357057 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbd1f1-1335-473a-8838-9c1abb879733 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:39.359724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-33cbd1f1-1335-473a-8838-9c1abb879733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:39.544104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbd1f1-1335-473a-8838-9c1abb879733 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:39.552548 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d34d6-3200-468a-9ed8-5a441d76a59f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:39.553391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d34d6-3200-468a-9ed8-5a441d76a59f service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:39.554658 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d34d6-3200-468a-9ed8-5a441d76a59f service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:39.554959 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d34d6-3200-468a-9ed8-5a441d76a59f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:39.576861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d34d6-3200-468a-9ed8-5a441d76a59f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:39.581502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d34d6-3200-468a-9ed8-5a441d76a59f service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:39.605890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 848/1658] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:39.890662 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bd17b-39b2-49d1-a1c8-20f312d9c205 None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:39.890943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bd17b-39b2-49d1-a1c8-20f312d9c205 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:39.895276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bd17b-39b2-49d1-a1c8-20f312d9c205 None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:39.896464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 811/1659] 10.222.0.25 () {54 vars in 960 bytes} [Mon May 9 18:29:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:29:39.902085 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8b942-4323-47df-b2dc-76ab0f47c004 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:39.902373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8b942-4323-47df-b2dc-76ab0f47c004 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:39.902876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8b942-4323-47df-b2dc-76ab0f47c004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:39.932254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c9c8b942-4323-47df-b2dc-76ab0f47c004 None None] MFA Rules not processed for user `f747dcb955fa499a972c46587d3db294`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:39.933489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c9c8b942-4323-47df-b2dc-76ab0f47c004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:40.057819 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 849/1660] 10.222.0.25 () {58 vars in 1057 bytes} [Mon May 9 18:29:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:40.068964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bd17b-39b2-49d1-a1c8-20f312d9c205 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:40.070260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8b942-4323-47df-b2dc-76ab0f47c004 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:40.078242 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c9c8b942-4323-47df-b2dc-76ab0f47c004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:40.115358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8b942-4323-47df-b2dc-76ab0f47c004 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:40.137086 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:40.137237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bd17b-39b2-49d1-a1c8-20f312d9c205 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:40.137697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:40.137697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:40.137697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] PATH_INFO: `/v3/projects/280795075c674e39b3a0e3c9fac9c47f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:40.139503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] RBAC: Authorizing `identity:delete_project(project_id=280795075c674e39b3a0e3c9fac9c47f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:40.140564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: WARNING py.warnings [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 18:29:40.140564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: warnings.warn(msg) May 09 18:29:40.140564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: May 09 18:29:40.150168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37001f38-a0c4-40f9-8cc2-baac5a1853d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f747dcb955fa499a972c46587d3db294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:40.152917 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37001f38-a0c4-40f9-8cc2-baac5a1853d6 service placement] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:40.153215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37001f38-a0c4-40f9-8cc2-baac5a1853d6 service placement] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:40.153500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37001f38-a0c4-40f9-8cc2-baac5a1853d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:40.155993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:40.160300 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-37001f38-a0c4-40f9-8cc2-baac5a1853d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:40.199110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37001f38-a0c4-40f9-8cc2-baac5a1853d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:40.201462 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37001f38-a0c4-40f9-8cc2-baac5a1853d6 service placement] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:40.226853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 812/1661] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:40.321144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '280795075c674e39b3a0e3c9fac9c47f', 'request_id': 'req-a7dab38b-6566-49e3-8663-7658f3bcf25b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:40.350911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '280795075c674e39b3a0e3c9fac9c47f', 'request_id': 'req-a7dab38b-6566-49e3-8663-7658f3bcf25b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:40.367582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '280795075c674e39b3a0e3c9fac9c47f', 'request_id': 'req-a7dab38b-6566-49e3-8663-7658f3bcf25b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:40.415998 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] The token cache is being invalidate because project 280795075c674e39b3a0e3c9fac9c47f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:29:40.416474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:40.417538 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:40.425592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 850/1662] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:29:40 2022] DELETE /identity/v3/projects/280795075c674e39b3a0e3c9fac9c47f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:29:40.427031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37001f38-a0c4-40f9-8cc2-baac5a1853d6 service placement] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:40.451262 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37001f38-a0c4-40f9-8cc2-baac5a1853d6 service placement] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:40.546927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:40.548440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:40.586415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01717f54-933d-4e59-bbdf-776fdaa7a3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:40.587585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01717f54-933d-4e59-bbdf-776fdaa7a3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:40.587882 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01717f54-933d-4e59-bbdf-776fdaa7a3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:40.588292 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01717f54-933d-4e59-bbdf-776fdaa7a3aa admin admin] PATH_INFO: `/v3/users/a328ccee066940be9832ffe20e61d741` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:40.589757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01717f54-933d-4e59-bbdf-776fdaa7a3aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a328ccee066940be9832ffe20e61d741)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:40.592327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01717f54-933d-4e59-bbdf-776fdaa7a3aa admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:40.745141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-01717f54-933d-4e59-bbdf-776fdaa7a3aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a328ccee066940be9832ffe20e61d741'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:40.762660 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-01717f54-933d-4e59-bbdf-776fdaa7a3aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a328ccee066940be9832ffe20e61d741'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:40.766246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-01717f54-933d-4e59-bbdf-776fdaa7a3aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a328ccee066940be9832ffe20e61d741'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:40.784626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-01717f54-933d-4e59-bbdf-776fdaa7a3aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a328ccee066940be9832ffe20e61d741'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:40.785291 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-01717f54-933d-4e59-bbdf-776fdaa7a3aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a328ccee066940be9832ffe20e61d741'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:40.820796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dab38b-6566-49e3-8663-7658f3bcf25b admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:40.833822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 813/1663] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:29:40 2022] DELETE /identity/v3/users/a328ccee066940be9832ffe20e61d741 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:40.850114 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592e3ca-091f-4296-806f-3914a7495a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:40.851023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592e3ca-091f-4296-806f-3914a7495a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:40.851342 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592e3ca-091f-4296-806f-3914a7495a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:40.851614 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592e3ca-091f-4296-806f-3914a7495a08 admin admin] PATH_INFO: `/v3/projects/31125b05c5e54ea28bac344b98366385` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:40.854359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592e3ca-091f-4296-806f-3914a7495a08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31125b05c5e54ea28bac344b98366385)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:40.860001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592e3ca-091f-4296-806f-3914a7495a08 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:40.961360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2592e3ca-091f-4296-806f-3914a7495a08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31125b05c5e54ea28bac344b98366385', 'request_id': 'req-2592e3ca-091f-4296-806f-3914a7495a08'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:40.981089 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2592e3ca-091f-4296-806f-3914a7495a08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31125b05c5e54ea28bac344b98366385', 'request_id': 'req-2592e3ca-091f-4296-806f-3914a7495a08'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:40.993149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2592e3ca-091f-4296-806f-3914a7495a08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31125b05c5e54ea28bac344b98366385', 'request_id': 'req-2592e3ca-091f-4296-806f-3914a7495a08'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:41.012253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2592e3ca-091f-4296-806f-3914a7495a08 admin admin] The token cache is being invalidate because project 31125b05c5e54ea28bac344b98366385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:29:41.012910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2592e3ca-091f-4296-806f-3914a7495a08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:41.013822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2592e3ca-091f-4296-806f-3914a7495a08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:41.020763 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 851/1664] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:29:40 2022] DELETE /identity/v3/projects/31125b05c5e54ea28bac344b98366385 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:29:41.025420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07163ab8-0ab6-496d-a943-7175eba663f3 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:41.025725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07163ab8-0ab6-496d-a943-7175eba663f3 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:41.026010 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07163ab8-0ab6-496d-a943-7175eba663f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:41.066737 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-07163ab8-0ab6-496d-a943-7175eba663f3 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:41.069711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-07163ab8-0ab6-496d-a943-7175eba663f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:41.267405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 814/1665] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:41.272192 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec72dfb-8bd1-4f7f-bbba-a5943c9cdfec None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:41.272551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec72dfb-8bd1-4f7f-bbba-a5943c9cdfec None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:41.272897 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec72dfb-8bd1-4f7f-bbba-a5943c9cdfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:41.305135 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-eec72dfb-8bd1-4f7f-bbba-a5943c9cdfec None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:41.306696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-eec72dfb-8bd1-4f7f-bbba-a5943c9cdfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:41.317865 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 852/1666] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:41.321617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07163ab8-0ab6-496d-a943-7175eba663f3 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:41.341051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07163ab8-0ab6-496d-a943-7175eba663f3 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:41.362486 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1591f7-02b9-4568-be8b-8363fa47c5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:41.363312 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1591f7-02b9-4568-be8b-8363fa47c5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:41.363629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1591f7-02b9-4568-be8b-8363fa47c5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:41.363944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1591f7-02b9-4568-be8b-8363fa47c5f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:41.364715 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1591f7-02b9-4568-be8b-8363fa47c5f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:29:41.365090 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1591f7-02b9-4568-be8b-8363fa47c5f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:41.366807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1591f7-02b9-4568-be8b-8363fa47c5f0 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:41.422455 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 815/1667] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:29:41.426770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec72dfb-8bd1-4f7f-bbba-a5943c9cdfec None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:41.438129 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec72dfb-8bd1-4f7f-bbba-a5943c9cdfec None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:41.462301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0176179-5ffe-4d84-bd99-842c828d7623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:41.463724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0176179-5ffe-4d84-bd99-842c828d7623 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:41.464005 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0176179-5ffe-4d84-bd99-842c828d7623 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:41.464747 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0176179-5ffe-4d84-bd99-842c828d7623 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:41.465088 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0176179-5ffe-4d84-bd99-842c828d7623 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:41.467070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0176179-5ffe-4d84-bd99-842c828d7623 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:41.541801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 853/1668] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:41 2022] POST /identity/v3/projects => generated 403 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:41.545866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1591f7-02b9-4568-be8b-8363fa47c5f0 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:41.558997 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1591f7-02b9-4568-be8b-8363fa47c5f0 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:41.703438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0176179-5ffe-4d84-bd99-842c828d7623 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:41.705132 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c0176179-5ffe-4d84-bd99-842c828d7623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:41.720610 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97d3f8-8e12-4395-ab40-5c3e3ed4ee00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:41.721303 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97d3f8-8e12-4395-ab40-5c3e3ed4ee00 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:41.721702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97d3f8-8e12-4395-ab40-5c3e3ed4ee00 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:41.722055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97d3f8-8e12-4395-ab40-5c3e3ed4ee00 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:41.722768 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97d3f8-8e12-4395-ab40-5c3e3ed4ee00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:41.736145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97d3f8-8e12-4395-ab40-5c3e3ed4ee00 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:41.789590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 816/1669] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:41 2022] POST /identity/v3/users => generated 302 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:29:41.795866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97d3f8-8e12-4395-ab40-5c3e3ed4ee00 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:41.830049 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97d3f8-8e12-4395-ab40-5c3e3ed4ee00 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:41.845852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108647-f40b-4ee1-a0b5-357474034db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:41.846846 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108647-f40b-4ee1-a0b5-357474034db5 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:41.850259 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108647-f40b-4ee1-a0b5-357474034db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:41.850686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108647-f40b-4ee1-a0b5-357474034db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:41.851852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a108647-f40b-4ee1-a0b5-357474034db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:41.858107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a108647-f40b-4ee1-a0b5-357474034db5 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:41.900731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0176179-5ffe-4d84-bd99-842c828d7623 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:41.903137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 817/1670] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:41.906572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108647-f40b-4ee1-a0b5-357474034db5 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:41.913438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:41.915864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:41.916200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:41.916483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] PATH_INFO: `/v3/projects/b9ed303802f64743b7c3f2790237fca4` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:41.920344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9ed303802f64743b7c3f2790237fca4)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:41.926681 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:41.933003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108647-f40b-4ee1-a0b5-357474034db5 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:41.944775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e3165-b2ee-44b4-b759-a9e506786d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:41.946181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e3165-b2ee-44b4-b759-a9e506786d4a admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:41.946478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e3165-b2ee-44b4-b759-a9e506786d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:41.946808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e3165-b2ee-44b4-b759-a9e506786d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:41.948339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e3165-b2ee-44b4-b759-a9e506786d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:41.952205 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e3165-b2ee-44b4-b759-a9e506786d4a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:41.969317 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 818/1671] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:41.976374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e3165-b2ee-44b4-b759-a9e506786d4a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:42.005440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e3165-b2ee-44b4-b759-a9e506786d4a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:42.019853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d425f-4ec6-4dfc-a36f-ce61aae7d807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:42.020772 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d425f-4ec6-4dfc-a36f-ce61aae7d807 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:42.021053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d425f-4ec6-4dfc-a36f-ce61aae7d807 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:42.021328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d425f-4ec6-4dfc-a36f-ce61aae7d807 admin admin] PATH_INFO: `/v3/projects/a0215e7469b54a89bb356d03a8c87aae/users/660bcaf429ca47819b07114143e9b415/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:42.072024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d425f-4ec6-4dfc-a36f-ce61aae7d807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0215e7469b54a89bb356d03a8c87aae, user_id=660bcaf429ca47819b07114143e9b415, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:42.077696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9ed303802f64743b7c3f2790237fca4', 'request_id': 'req-374d6990-0368-4ae3-a78b-2c63407394c4'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.079816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d425f-4ec6-4dfc-a36f-ce61aae7d807 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:42.085575 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9ed303802f64743b7c3f2790237fca4', 'request_id': 'req-374d6990-0368-4ae3-a78b-2c63407394c4'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.099163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9ed303802f64743b7c3f2790237fca4', 'request_id': 'req-374d6990-0368-4ae3-a78b-2c63407394c4'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.107567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 819/1672] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:41 2022] PUT /identity/v3/projects/a0215e7469b54a89bb356d03a8c87aae/users/660bcaf429ca47819b07114143e9b415/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:42.120493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] The token cache is being invalidate because project b9ed303802f64743b7c3f2790237fca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:29:42.120916 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7e865-96ea-4068-afbf-4f7cfd86cf6f None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:42.121222 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7e865-96ea-4068-afbf-4f7cfd86cf6f None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:42.121621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.121737 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7e865-96ea-4068-afbf-4f7cfd86cf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:42.123973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.130382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 854/1673] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:29:41 2022] DELETE /identity/v3/projects/b9ed303802f64743b7c3f2790237fca4 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:29:42.134754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:42.137163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:42.188209 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-bea7e865-96ea-4068-afbf-4f7cfd86cf6f None None] MFA Rules not processed for user `660bcaf429ca47819b07114143e9b415`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:42.193299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-bea7e865-96ea-4068-afbf-4f7cfd86cf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:42.239980 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d6990-0368-4ae3-a78b-2c63407394c4 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:42.371452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 820/1674] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:42.372228 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7e865-96ea-4068-afbf-4f7cfd86cf6f None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:42.373677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-bea7e865-96ea-4068-afbf-4f7cfd86cf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:42.444225 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd258a70-9494-4f73-9afb-97715c22c8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:42.449548 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd258a70-9494-4f73-9afb-97715c22c8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:42.449838 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd258a70-9494-4f73-9afb-97715c22c8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:42.450120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd258a70-9494-4f73-9afb-97715c22c8ea admin admin] PATH_INFO: `/v3/users/3620df13ffeb426497723934c8a26626` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:42.455062 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd258a70-9494-4f73-9afb-97715c22c8ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=3620df13ffeb426497723934c8a26626)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:42.457089 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd258a70-9494-4f73-9afb-97715c22c8ea admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:42.528957 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7e865-96ea-4068-afbf-4f7cfd86cf6f None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:42.564449 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232a094-e003-406d-adb3-3ad0f762391c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:42.569401 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232a094-e003-406d-adb3-3ad0f762391c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:42.569689 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232a094-e003-406d-adb3-3ad0f762391c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:42.572176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232a094-e003-406d-adb3-3ad0f762391c admin admin] PATH_INFO: `/v3/projects/e05dfefe5f6749e38a51fda868ceb829` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:42.577689 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232a094-e003-406d-adb3-3ad0f762391c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e05dfefe5f6749e38a51fda868ceb829)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:42.581535 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232a094-e003-406d-adb3-3ad0f762391c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:42.647096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-bd258a70-9494-4f73-9afb-97715c22c8ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3620df13ffeb426497723934c8a26626'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.649934 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-bd258a70-9494-4f73-9afb-97715c22c8ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3620df13ffeb426497723934c8a26626'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.685134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-bd258a70-9494-4f73-9afb-97715c22c8ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3620df13ffeb426497723934c8a26626'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.705662 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-bd258a70-9494-4f73-9afb-97715c22c8ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3620df13ffeb426497723934c8a26626'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.710803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-bd258a70-9494-4f73-9afb-97715c22c8ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3620df13ffeb426497723934c8a26626'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.717850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8232a094-e003-406d-adb3-3ad0f762391c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e05dfefe5f6749e38a51fda868ceb829', 'request_id': 'req-8232a094-e003-406d-adb3-3ad0f762391c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.725403 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8232a094-e003-406d-adb3-3ad0f762391c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e05dfefe5f6749e38a51fda868ceb829', 'request_id': 'req-8232a094-e003-406d-adb3-3ad0f762391c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.736876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8232a094-e003-406d-adb3-3ad0f762391c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e05dfefe5f6749e38a51fda868ceb829', 'request_id': 'req-8232a094-e003-406d-adb3-3ad0f762391c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.768585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 855/1675] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:29:42 2022] DELETE /identity/v3/users/3620df13ffeb426497723934c8a26626 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:29:42.768585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd258a70-9494-4f73-9afb-97715c22c8ea admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:42.771969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-bd258a70-9494-4f73-9afb-97715c22c8ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:42.789413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8232a094-e003-406d-adb3-3ad0f762391c admin admin] The token cache is being invalidate because project e05dfefe5f6749e38a51fda868ceb829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:29:42.789704 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8232a094-e003-406d-adb3-3ad0f762391c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.790342 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8232a094-e003-406d-adb3-3ad0f762391c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:42.792260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 821/1676] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:29:42 2022] DELETE /identity/v3/projects/e05dfefe5f6749e38a51fda868ceb829 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:42.802890 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446f1c7-2c85-44cb-a6ec-3fda20cbe03f None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:42.803295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446f1c7-2c85-44cb-a6ec-3fda20cbe03f None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:42.803644 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446f1c7-2c85-44cb-a6ec-3fda20cbe03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:42.887519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd258a70-9494-4f73-9afb-97715c22c8ea admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:42.888308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-b446f1c7-2c85-44cb-a6ec-3fda20cbe03f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:42.891451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b446f1c7-2c85-44cb-a6ec-3fda20cbe03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:42.898641 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e986f1-6509-43a7-8c7a-4f4fbba94e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:42.899375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e986f1-6509-43a7-8c7a-4f4fbba94e78 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:42.900985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e986f1-6509-43a7-8c7a-4f4fbba94e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:42.900985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e986f1-6509-43a7-8c7a-4f4fbba94e78 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:42.900985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e986f1-6509-43a7-8c7a-4f4fbba94e78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:42.904424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e986f1-6509-43a7-8c7a-4f4fbba94e78 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:42.929542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 822/1677] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:42.940093 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a89a0-ac17-4186-a5b4-e40c68c4c572 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:42.940381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a89a0-ac17-4186-a5b4-e40c68c4c572 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:42.940652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a89a0-ac17-4186-a5b4-e40c68c4c572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:42.945874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 856/1678] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:42 2022] POST /identity/v3/projects => generated 403 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:42.950097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e986f1-6509-43a7-8c7a-4f4fbba94e78 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:42.952099 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e3e986f1-6509-43a7-8c7a-4f4fbba94e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:42.993377 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ac9a89a0-ac17-4186-a5b4-e40c68c4c572 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:42.994610 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ac9a89a0-ac17-4186-a5b4-e40c68c4c572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:43.145837 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e986f1-6509-43a7-8c7a-4f4fbba94e78 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:43.162616 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1334c9e-7f06-4f47-aa81-e3d32783081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:43.163571 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1334c9e-7f06-4f47-aa81-e3d32783081f admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:43.168016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1334c9e-7f06-4f47-aa81-e3d32783081f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:43.168452 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1334c9e-7f06-4f47-aa81-e3d32783081f admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:43.169372 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1334c9e-7f06-4f47-aa81-e3d32783081f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:43.176627 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1334c9e-7f06-4f47-aa81-e3d32783081f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:43.234257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 857/1679] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:42 2022] POST /identity/v3/users => generated 308 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:43.239191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1334c9e-7f06-4f47-aa81-e3d32783081f admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:43.261785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1334c9e-7f06-4f47-aa81-e3d32783081f admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:43.281763 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b7e42-4080-4541-bd99-a2739e98cf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:43.282697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b7e42-4080-4541-bd99-a2739e98cf0b admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:43.283659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b7e42-4080-4541-bd99-a2739e98cf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:43.284074 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b7e42-4080-4541-bd99-a2739e98cf0b admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:43.285321 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b7e42-4080-4541-bd99-a2739e98cf0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:43.288697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b7e42-4080-4541-bd99-a2739e98cf0b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:43.324711 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 858/1680] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:29:43.329068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b7e42-4080-4541-bd99-a2739e98cf0b admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:43.367436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b7e42-4080-4541-bd99-a2739e98cf0b admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:43.379560 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9bcdf-7348-4b76-a8e4-48ec35ad578f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:43.383523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9bcdf-7348-4b76-a8e4-48ec35ad578f admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:43.395676 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9bcdf-7348-4b76-a8e4-48ec35ad578f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:43.395972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9bcdf-7348-4b76-a8e4-48ec35ad578f admin admin] PATH_INFO: `/v3/projects/099b98b265a34a7ca16ac33f9b9890ae/users/ac9e6b1aa85a4ec8a0bf08cdab75082d/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:43.416837 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9bcdf-7348-4b76-a8e4-48ec35ad578f admin admin] RBAC: Authorizing `identity:create_grant(project_id=099b98b265a34a7ca16ac33f9b9890ae, user_id=ac9e6b1aa85a4ec8a0bf08cdab75082d, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:43.419482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 823/1681] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:43.424062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a89a0-ac17-4186-a5b4-e40c68c4c572 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:43.427050 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9bcdf-7348-4b76-a8e4-48ec35ad578f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:43.444569 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a89a0-ac17-4186-a5b4-e40c68c4c572 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:43.451889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 859/1682] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:43 2022] PUT /identity/v3/projects/099b98b265a34a7ca16ac33f9b9890ae/users/ac9e6b1aa85a4ec8a0bf08cdab75082d/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:43.452924 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9bcdf-7348-4b76-a8e4-48ec35ad578f admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:43.583518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfd568-9595-4bfc-80ba-110d0de9d19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:43.584216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfd568-9595-4bfc-80ba-110d0de9d19c admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:43.584501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfd568-9595-4bfc-80ba-110d0de9d19c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:43.584775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfd568-9595-4bfc-80ba-110d0de9d19c admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:43.585637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfd568-9595-4bfc-80ba-110d0de9d19c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:29:43.585805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfd568-9595-4bfc-80ba-110d0de9d19c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:43.587431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfd568-9595-4bfc-80ba-110d0de9d19c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:43.608431 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9bcdf-7348-4b76-a8e4-48ec35ad578f admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:43.609698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 824/1683] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:29:43.613915 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfd568-9595-4bfc-80ba-110d0de9d19c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:43.633876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfd568-9595-4bfc-80ba-110d0de9d19c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:43.643702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6650bc-e4c5-4fa1-8c74-af3fdae18fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:43.646344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6650bc-e4c5-4fa1-8c74-af3fdae18fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:43.646652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e614278-c54e-41de-8647-a3e85fcd61a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:43.647355 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e614278-c54e-41de-8647-a3e85fcd61a2 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:43.647639 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e614278-c54e-41de-8647-a3e85fcd61a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:43.647908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e614278-c54e-41de-8647-a3e85fcd61a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:43.648594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e614278-c54e-41de-8647-a3e85fcd61a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:43.649515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6650bc-e4c5-4fa1-8c74-af3fdae18fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:43.649834 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6650bc-e4c5-4fa1-8c74-af3fdae18fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:43.650486 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e614278-c54e-41de-8647-a3e85fcd61a2 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:43.659382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6650bc-e4c5-4fa1-8c74-af3fdae18fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:43.661037 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6650bc-e4c5-4fa1-8c74-af3fdae18fd1 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:43.709864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 860/1684] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:43.711908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6650bc-e4c5-4fa1-8c74-af3fdae18fd1 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:43.735399 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6650bc-e4c5-4fa1-8c74-af3fdae18fd1 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:43.741170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 825/1685] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:43 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:43.745113 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e614278-c54e-41de-8647-a3e85fcd61a2 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:43.927390 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec11d3-e57c-412e-8c07-0cdd6538727d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:43.928304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec11d3-e57c-412e-8c07-0cdd6538727d admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:43.928584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec11d3-e57c-412e-8c07-0cdd6538727d admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:43.928858 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec11d3-e57c-412e-8c07-0cdd6538727d admin admin] PATH_INFO: `/v3/projects/099b98b265a34a7ca16ac33f9b9890ae/users/ac9e6b1aa85a4ec8a0bf08cdab75082d/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:43.931109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec11d3-e57c-412e-8c07-0cdd6538727d admin admin] RBAC: Authorizing `identity:create_grant(project_id=099b98b265a34a7ca16ac33f9b9890ae, user_id=ac9e6b1aa85a4ec8a0bf08cdab75082d, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:43.933813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec11d3-e57c-412e-8c07-0cdd6538727d admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:43.976577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 861/1686] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:43 2022] PUT /identity/v3/projects/099b98b265a34a7ca16ac33f9b9890ae/users/ac9e6b1aa85a4ec8a0bf08cdab75082d/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:43.983319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e614278-c54e-41de-8647-a3e85fcd61a2 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:43.985813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc0c14-6a3a-4e3d-a49c-d72586bacc5a None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:43.986252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc0c14-6a3a-4e3d-a49c-d72586bacc5a None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:43.986661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc0c14-6a3a-4e3d-a49c-d72586bacc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:44.053877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-95dc0c14-6a3a-4e3d-a49c-d72586bacc5a None None] MFA Rules not processed for user `ac9e6b1aa85a4ec8a0bf08cdab75082d`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:44.059234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-95dc0c14-6a3a-4e3d-a49c-d72586bacc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:44.148905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc030ef-0a94-4614-9715-cbe6d97aaf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:44.149554 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc030ef-0a94-4614-9715-cbe6d97aaf38 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:44.150039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc030ef-0a94-4614-9715-cbe6d97aaf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:44.150148 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc030ef-0a94-4614-9715-cbe6d97aaf38 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:44.151030 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc030ef-0a94-4614-9715-cbe6d97aaf38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:44.152745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc030ef-0a94-4614-9715-cbe6d97aaf38 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:44.223276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 826/1687] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:43 2022] POST /identity/v3/users => generated 294 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:44.228931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc030ef-0a94-4614-9715-cbe6d97aaf38 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:44.240918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 862/1688] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:43 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:44.247031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bbcfd-02fc-4af1-b186-ba5d7f2bbf54 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:44.251470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bbcfd-02fc-4af1-b186-ba5d7f2bbf54 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:44.251843 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bbcfd-02fc-4af1-b186-ba5d7f2bbf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:44.269661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc030ef-0a94-4614-9715-cbe6d97aaf38 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:44.279227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb60f71-5051-424b-b956-ff331ed20985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:44.280352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb60f71-5051-424b-b956-ff331ed20985 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:44.280939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb60f71-5051-424b-b956-ff331ed20985 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:44.281729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb60f71-5051-424b-b956-ff331ed20985 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:44.285589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb60f71-5051-424b-b956-ff331ed20985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:44.290932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-a97bbcfd-02fc-4af1-b186-ba5d7f2bbf54 None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:44.292766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a97bbcfd-02fc-4af1-b186-ba5d7f2bbf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:44.298901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb60f71-5051-424b-b956-ff331ed20985 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:44.347385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 827/1689] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:44.349856 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb60f71-5051-424b-b956-ff331ed20985 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:44.352484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7eb60f71-5051-424b-b956-ff331ed20985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:44.414120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 863/1690] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:29:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:44.414906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97bbcfd-02fc-4af1-b186-ba5d7f2bbf54 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:44.422172 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a97bbcfd-02fc-4af1-b186-ba5d7f2bbf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:44.455210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97bbcfd-02fc-4af1-b186-ba5d7f2bbf54 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:44.470020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:44.470485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:44.470761 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:44.471031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] PATH_INFO: `/v3/projects/7487ca4e595a4b058f99731ff57cb1cf` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:44.472633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7487ca4e595a4b058f99731ff57cb1cf)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:44.474543 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:44.507795 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb60f71-5051-424b-b956-ff331ed20985 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:44.523125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7566470-591c-4d14-8c72-a8d29ee1fb57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:44.523828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7566470-591c-4d14-8c72-a8d29ee1fb57 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:44.524107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7566470-591c-4d14-8c72-a8d29ee1fb57 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:44.524380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7566470-591c-4d14-8c72-a8d29ee1fb57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:44.540818 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7566470-591c-4d14-8c72-a8d29ee1fb57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:44.543656 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7566470-591c-4d14-8c72-a8d29ee1fb57 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:44.579270 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 828/1691] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:44 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:44.580075 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7566470-591c-4d14-8c72-a8d29ee1fb57 service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:44.604959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7566470-591c-4d14-8c72-a8d29ee1fb57 service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:44.635759 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7487ca4e595a4b058f99731ff57cb1cf', 'request_id': 'req-a07d7117-7915-4c21-8353-af4453f1b919'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:44.698984 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7487ca4e595a4b058f99731ff57cb1cf', 'request_id': 'req-a07d7117-7915-4c21-8353-af4453f1b919'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:44.722738 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7487ca4e595a4b058f99731ff57cb1cf', 'request_id': 'req-a07d7117-7915-4c21-8353-af4453f1b919'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:44.788405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] The token cache is being invalidate because project 7487ca4e595a4b058f99731ff57cb1cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:29:44.789961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:44.790794 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221831b3-3217-45be-95e8-6d45764ee31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:44.791800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221831b3-3217-45be-95e8-6d45764ee31e admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:44.792108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221831b3-3217-45be-95e8-6d45764ee31e admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:44.792470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221831b3-3217-45be-95e8-6d45764ee31e admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:44.793115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221831b3-3217-45be-95e8-6d45764ee31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:44.793228 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:44.794718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221831b3-3217-45be-95e8-6d45764ee31e admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:44.801939 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 864/1692] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:29:44 2022] DELETE /identity/v3/projects/7487ca4e595a4b058f99731ff57cb1cf => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:44.803741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:44.805534 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:44.832085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 829/1693] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:44.832085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6933688-ac7a-4e8e-b0f3-391712f34441 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:44.832085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6933688-ac7a-4e8e-b0f3-391712f34441 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:44.832085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6933688-ac7a-4e8e-b0f3-391712f34441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:44.893039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f6933688-ac7a-4e8e-b0f3-391712f34441 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:44.894261 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f6933688-ac7a-4e8e-b0f3-391712f34441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:44.904576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 830/1694] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:44.906020 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6933688-ac7a-4e8e-b0f3-391712f34441 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:44.907365 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f6933688-ac7a-4e8e-b0f3-391712f34441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:44.935419 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6933688-ac7a-4e8e-b0f3-391712f34441 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:44.948705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8594c6-c585-40f9-8d66-c6c10a650a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:44.951536 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8594c6-c585-40f9-8d66-c6c10a650a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:44.954111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8594c6-c585-40f9-8d66-c6c10a650a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:44.954552 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8594c6-c585-40f9-8d66-c6c10a650a5d admin admin] PATH_INFO: `/v3/projects/2596302735dd4b07b3d9ca4de4591ed6/users/424bba92b5554defaa145a078f762cf0/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:44.962718 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d7117-7915-4c21-8353-af4453f1b919 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:44.977485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01fc81-67c2-4437-a691-2712a6faee7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:44.978379 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01fc81-67c2-4437-a691-2712a6faee7f service swift] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:44.978741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01fc81-67c2-4437-a691-2712a6faee7f service swift] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:44.979125 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01fc81-67c2-4437-a691-2712a6faee7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:44.981910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-8b01fc81-67c2-4437-a691-2712a6faee7f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:44.996994 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8594c6-c585-40f9-8d66-c6c10a650a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2596302735dd4b07b3d9ca4de4591ed6, user_id=424bba92b5554defaa145a078f762cf0, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:45.000333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8594c6-c585-40f9-8d66-c6c10a650a5d admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:45.033775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 831/1695] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:44 2022] PUT /identity/v3/projects/2596302735dd4b07b3d9ca4de4591ed6/users/424bba92b5554defaa145a078f762cf0/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:45.036010 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd33334-3953-4fac-b81e-f4105ee5e987 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:45.036680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd33334-3953-4fac-b81e-f4105ee5e987 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:45.037285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd33334-3953-4fac-b81e-f4105ee5e987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:45.094535 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-afd33334-3953-4fac-b81e-f4105ee5e987 None None] MFA Rules not processed for user `f99208187e4e4d2eb9f52107ae8f50f2`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:45.095903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-afd33334-3953-4fac-b81e-f4105ee5e987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:45.125571 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b01fc81-67c2-4437-a691-2712a6faee7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:45.129173 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b01fc81-67c2-4437-a691-2712a6faee7f service swift] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:45.144777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 865/1696] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:29:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 18:29:45.146553 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c00fd-006c-4528-a9e3-b85d9d5e0560 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:45.146982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c00fd-006c-4528-a9e3-b85d9d5e0560 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:45.147322 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c00fd-006c-4528-a9e3-b85d9d5e0560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:45.172966 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b95c00fd-006c-4528-a9e3-b85d9d5e0560 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:45.177871 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b95c00fd-006c-4528-a9e3-b85d9d5e0560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:45.317087 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 832/1697] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:45.325705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd33334-3953-4fac-b81e-f4105ee5e987 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:45.330699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-afd33334-3953-4fac-b81e-f4105ee5e987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:45.396582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 866/1698] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:45.401179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702382f1-8364-446b-99e9-92e304543a29 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:45.401490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702382f1-8364-446b-99e9-92e304543a29 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:45.401931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702382f1-8364-446b-99e9-92e304543a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:45.512740 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-702382f1-8364-446b-99e9-92e304543a29 None None] MFA Rules not processed for user `424bba92b5554defaa145a078f762cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:45.516556 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-702382f1-8364-446b-99e9-92e304543a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:45.531859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd33334-3953-4fac-b81e-f4105ee5e987 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:45.549333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514c2cf-7721-4600-b554-e92fbe71a3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:45.550469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514c2cf-7721-4600-b554-e92fbe71a3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:45.553479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514c2cf-7721-4600-b554-e92fbe71a3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:45.553761 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514c2cf-7721-4600-b554-e92fbe71a3bb admin admin] PATH_INFO: `/v3/users/660bcaf429ca47819b07114143e9b415` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:45.560904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514c2cf-7721-4600-b554-e92fbe71a3bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=660bcaf429ca47819b07114143e9b415)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:45.560904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514c2cf-7721-4600-b554-e92fbe71a3bb admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:45.737778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e514c2cf-7721-4600-b554-e92fbe71a3bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '660bcaf429ca47819b07114143e9b415'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:45.746661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 867/1699] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:45.749708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702382f1-8364-446b-99e9-92e304543a29 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:45.754450 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-702382f1-8364-446b-99e9-92e304543a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:45.761552 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e514c2cf-7721-4600-b554-e92fbe71a3bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '660bcaf429ca47819b07114143e9b415'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:45.763714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e514c2cf-7721-4600-b554-e92fbe71a3bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '660bcaf429ca47819b07114143e9b415'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:45.782374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e514c2cf-7721-4600-b554-e92fbe71a3bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '660bcaf429ca47819b07114143e9b415'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:45.788805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e514c2cf-7721-4600-b554-e92fbe71a3bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '660bcaf429ca47819b07114143e9b415'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:45.810483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702382f1-8364-446b-99e9-92e304543a29 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:45.837097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24aac6-2c31-4b85-9dec-fffeda0657ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:45.843127 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24aac6-2c31-4b85-9dec-fffeda0657ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:45.843514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24aac6-2c31-4b85-9dec-fffeda0657ec admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:45.843812 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24aac6-2c31-4b85-9dec-fffeda0657ec admin admin] PATH_INFO: `/v3/users/9e73b40a50c1493bbee6142044605826` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:45.847932 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 833/1700] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:29:45 2022] DELETE /identity/v3/users/660bcaf429ca47819b07114143e9b415 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:45.847932 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514c2cf-7721-4600-b554-e92fbe71a3bb admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:45.849658 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24aac6-2c31-4b85-9dec-fffeda0657ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e73b40a50c1493bbee6142044605826)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:45.850566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e514c2cf-7721-4600-b554-e92fbe71a3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:45.859823 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24aac6-2c31-4b85-9dec-fffeda0657ec admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:46.017921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ad24aac6-2c31-4b85-9dec-fffeda0657ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e73b40a50c1493bbee6142044605826'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:46.019028 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ad24aac6-2c31-4b85-9dec-fffeda0657ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e73b40a50c1493bbee6142044605826'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:46.036407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ad24aac6-2c31-4b85-9dec-fffeda0657ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e73b40a50c1493bbee6142044605826'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:46.053344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ad24aac6-2c31-4b85-9dec-fffeda0657ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e73b40a50c1493bbee6142044605826'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:46.055843 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ad24aac6-2c31-4b85-9dec-fffeda0657ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e73b40a50c1493bbee6142044605826'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:46.057127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514c2cf-7721-4600-b554-e92fbe71a3bb admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:46.084144 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5302a-47ef-432c-b5e8-82c111f9426b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:46.085007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5302a-47ef-432c-b5e8-82c111f9426b service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:46.085398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5302a-47ef-432c-b5e8-82c111f9426b service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:46.085777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5302a-47ef-432c-b5e8-82c111f9426b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:46.094214 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-5bf5302a-47ef-432c-b5e8-82c111f9426b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:46.112250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 868/1701] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:29:45 2022] DELETE /identity/v3/users/9e73b40a50c1493bbee6142044605826 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:46.114867 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24aac6-2c31-4b85-9dec-fffeda0657ec admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:46.120453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ad24aac6-2c31-4b85-9dec-fffeda0657ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:46.256266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5302a-47ef-432c-b5e8-82c111f9426b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:46.258742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5302a-47ef-432c-b5e8-82c111f9426b service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:46.285964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 834/1702] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:46.287071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5302a-47ef-432c-b5e8-82c111f9426b service nova] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:46.288668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-5bf5302a-47ef-432c-b5e8-82c111f9426b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:46.332020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24aac6-2c31-4b85-9dec-fffeda0657ec admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:46.351430 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d56cd1-8bae-4c1b-b037-a1dd4b68d0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:46.352151 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d56cd1-8bae-4c1b-b037-a1dd4b68d0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:46.352438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d56cd1-8bae-4c1b-b037-a1dd4b68d0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:46.352715 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d56cd1-8bae-4c1b-b037-a1dd4b68d0c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:46.353464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d56cd1-8bae-4c1b-b037-a1dd4b68d0c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:29:46.355613 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d56cd1-8bae-4c1b-b037-a1dd4b68d0c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:46.357292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d56cd1-8bae-4c1b-b037-a1dd4b68d0c8 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:46.411348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 869/1703] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:29:46.412861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a66302-1a60-4325-98c9-4a96bfbcc3ee None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:46.413151 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a66302-1a60-4325-98c9-4a96bfbcc3ee None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:46.413492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a66302-1a60-4325-98c9-4a96bfbcc3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:46.465331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5302a-47ef-432c-b5e8-82c111f9426b service nova] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:46.475980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c132704-50fc-44b9-9d88-ca4896f2eebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:46.478096 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c132704-50fc-44b9-9d88-ca4896f2eebb admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:46.478492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c132704-50fc-44b9-9d88-ca4896f2eebb admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:46.478863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c132704-50fc-44b9-9d88-ca4896f2eebb admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:46.479724 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c132704-50fc-44b9-9d88-ca4896f2eebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:46.481872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c132704-50fc-44b9-9d88-ca4896f2eebb admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:46.507016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-f4a66302-1a60-4325-98c9-4a96bfbcc3ee None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:46.508468 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f4a66302-1a60-4325-98c9-4a96bfbcc3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:46.551544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 870/1704] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:46.555514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c66d7-c016-4220-854f-5f3a8fc192a0 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:46.556018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c66d7-c016-4220-854f-5f3a8fc192a0 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:46.556815 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c66d7-c016-4220-854f-5f3a8fc192a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:46.567837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 835/1705] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:46 2022] POST /identity/v3/projects => generated 389 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:46.568864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c132704-50fc-44b9-9d88-ca4896f2eebb admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:46.605033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-828c66d7-c016-4220-854f-5f3a8fc192a0 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:46.609196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-828c66d7-c016-4220-854f-5f3a8fc192a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:46.726734 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 871/1706] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:46.728267 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c66d7-c016-4220-854f-5f3a8fc192a0 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:46.757051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c132704-50fc-44b9-9d88-ca4896f2eebb admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:46.768021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c66d7-c016-4220-854f-5f3a8fc192a0 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:46.769628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0883483-e812-4e30-a283-5ea23f423e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:46.770428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0883483-e812-4e30-a283-5ea23f423e84 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:46.770819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0883483-e812-4e30-a283-5ea23f423e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:46.771211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0883483-e812-4e30-a283-5ea23f423e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:46.772021 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0883483-e812-4e30-a283-5ea23f423e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:46.774057 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0883483-e812-4e30-a283-5ea23f423e84 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:46.781187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad18d21-3893-4b94-b0e6-184c014a4520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:46.785799 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad18d21-3893-4b94-b0e6-184c014a4520 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:46.786105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad18d21-3893-4b94-b0e6-184c014a4520 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:46.786389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad18d21-3893-4b94-b0e6-184c014a4520 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:46.789169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad18d21-3893-4b94-b0e6-184c014a4520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:46.794148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad18d21-3893-4b94-b0e6-184c014a4520 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:46.842450 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 872/1707] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:46 2022] POST /identity/v3/users => generated 301 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:46.843945 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad18d21-3893-4b94-b0e6-184c014a4520 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:46.851109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 836/1708] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:46 2022] POST /identity/v3/projects => generated 369 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:46.852215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0883483-e812-4e30-a283-5ea23f423e84 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:46.855282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4ad18d21-3893-4b94-b0e6-184c014a4520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:47.008785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad18d21-3893-4b94-b0e6-184c014a4520 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:47.055558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0883483-e812-4e30-a283-5ea23f423e84 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:47.074921 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80b456-3bdd-4407-85d2-3e7cc035a3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:47.075907 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80b456-3bdd-4407-85d2-3e7cc035a3ae admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:47.076295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80b456-3bdd-4407-85d2-3e7cc035a3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:47.076677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80b456-3bdd-4407-85d2-3e7cc035a3ae admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:47.078056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80b456-3bdd-4407-85d2-3e7cc035a3ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:47.094774 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80b456-3bdd-4407-85d2-3e7cc035a3ae admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:47.134125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 837/1709] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:47.135137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80b456-3bdd-4407-85d2-3e7cc035a3ae admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:47.165771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca10ebf-c74e-41e1-b5ad-39889cdbaae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:47.166625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca10ebf-c74e-41e1-b5ad-39889cdbaae3 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:47.166951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca10ebf-c74e-41e1-b5ad-39889cdbaae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:47.167302 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca10ebf-c74e-41e1-b5ad-39889cdbaae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:47.176766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80b456-3bdd-4407-85d2-3e7cc035a3ae admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:47.188347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca10ebf-c74e-41e1-b5ad-39889cdbaae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:47.190687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca10ebf-c74e-41e1-b5ad-39889cdbaae3 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:47.198072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bd430-6205-4b12-89bb-bfba86ced97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:47.199518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bd430-6205-4b12-89bb-bfba86ced97d admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:47.202958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bd430-6205-4b12-89bb-bfba86ced97d admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:47.203582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bd430-6205-4b12-89bb-bfba86ced97d admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:47.205710 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46bd430-6205-4b12-89bb-bfba86ced97d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:47.218807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46bd430-6205-4b12-89bb-bfba86ced97d admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:47.220636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 873/1710] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:47.221507 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca10ebf-c74e-41e1-b5ad-39889cdbaae3 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:47.232145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7ca10ebf-c74e-41e1-b5ad-39889cdbaae3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:47.266125 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca10ebf-c74e-41e1-b5ad-39889cdbaae3 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:47.278430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 838/1711] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:47 2022] POST /identity/v3/users => generated 285 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:47.279720 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bd430-6205-4b12-89bb-bfba86ced97d admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:47.284228 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f890e-ee20-4bb3-b9e0-056e0207af52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:47.284862 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f890e-ee20-4bb3-b9e0-056e0207af52 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:47.285145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f890e-ee20-4bb3-b9e0-056e0207af52 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:47.285422 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f890e-ee20-4bb3-b9e0-056e0207af52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:47.310458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bd430-6205-4b12-89bb-bfba86ced97d admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:47.317368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5531d7-3524-424d-a64f-2440f9d10ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:47.318623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5531d7-3524-424d-a64f-2440f9d10ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:47.319062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5531d7-3524-424d-a64f-2440f9d10ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:47.319526 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5531d7-3524-424d-a64f-2440f9d10ee3 admin admin] PATH_INFO: `/v3/projects/e2c4bd2ed1ce46fe8bc9ed3e4a565f53/users/5267052717f441ebb8fc063f4ad4a26f/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:47.329418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f890e-ee20-4bb3-b9e0-056e0207af52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:47.330977 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f890e-ee20-4bb3-b9e0-056e0207af52 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:47.360361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5531d7-3524-424d-a64f-2440f9d10ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2c4bd2ed1ce46fe8bc9ed3e4a565f53, user_id=5267052717f441ebb8fc063f4ad4a26f, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:47.362972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5531d7-3524-424d-a64f-2440f9d10ee3 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:47.380629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 874/1712] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:47.381550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f890e-ee20-4bb3-b9e0-056e0207af52 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:47.418698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 839/1713] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:47 2022] PUT /identity/v3/projects/e2c4bd2ed1ce46fe8bc9ed3e4a565f53/users/5267052717f441ebb8fc063f4ad4a26f/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:47.420979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5531d7-3524-424d-a64f-2440f9d10ee3 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:47.422637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f890e-ee20-4bb3-b9e0-056e0207af52 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:47.589963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861c9a3-bf81-440e-bef1-d18a1a7988ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:47.590829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861c9a3-bf81-440e-bef1-d18a1a7988ca admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:47.591676 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861c9a3-bf81-440e-bef1-d18a1a7988ca admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:47.592074 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861c9a3-bf81-440e-bef1-d18a1a7988ca admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:47.596317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861c9a3-bf81-440e-bef1-d18a1a7988ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:47.598089 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861c9a3-bf81-440e-bef1-d18a1a7988ca admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:47.603588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5531d7-3524-424d-a64f-2440f9d10ee3 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:47.617419 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 875/1714] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:47.620604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861c9a3-bf81-440e-bef1-d18a1a7988ca admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:47.622401 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974a44e-a4ee-4336-9b5f-9b8371b126e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:47.626137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974a44e-a4ee-4336-9b5f-9b8371b126e8 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:47.626455 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974a44e-a4ee-4336-9b5f-9b8371b126e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:47.626746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974a44e-a4ee-4336-9b5f-9b8371b126e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:47.629822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974a44e-a4ee-4336-9b5f-9b8371b126e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:47.635683 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861c9a3-bf81-440e-bef1-d18a1a7988ca admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:47.636160 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974a44e-a4ee-4336-9b5f-9b8371b126e8 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:47.659733 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa2652-1bb9-462b-86d7-b6df016e1015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:47.660580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa2652-1bb9-462b-86d7-b6df016e1015 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:47.660943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa2652-1bb9-462b-86d7-b6df016e1015 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:47.661420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa2652-1bb9-462b-86d7-b6df016e1015 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:47.662235 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa2652-1bb9-462b-86d7-b6df016e1015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:47.664023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa2652-1bb9-462b-86d7-b6df016e1015 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:47.682235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 840/1715] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:47.688151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974a44e-a4ee-4336-9b5f-9b8371b126e8 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:47.695348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 876/1716] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:47.696522 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa2652-1bb9-462b-86d7-b6df016e1015 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:47.730537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974a44e-a4ee-4336-9b5f-9b8371b126e8 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:47.752665 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa2652-1bb9-462b-86d7-b6df016e1015 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:47.753198 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cc55c-7fc2-455f-a1a0-f48173a3dfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:47.754192 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cc55c-7fc2-455f-a1a0-f48173a3dfde admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:47.754481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cc55c-7fc2-455f-a1a0-f48173a3dfde admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:47.754757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cc55c-7fc2-455f-a1a0-f48173a3dfde admin admin] PATH_INFO: `/v3/projects/e2c4bd2ed1ce46fe8bc9ed3e4a565f53/users/5267052717f441ebb8fc063f4ad4a26f/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:47.759072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0cc55c-7fc2-455f-a1a0-f48173a3dfde admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2c4bd2ed1ce46fe8bc9ed3e4a565f53, user_id=5267052717f441ebb8fc063f4ad4a26f, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:47.773817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0cc55c-7fc2-455f-a1a0-f48173a3dfde admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:47.789246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23495eaf-ab25-4347-8780-49c69e46565a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:47.793842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23495eaf-ab25-4347-8780-49c69e46565a admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:47.796605 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23495eaf-ab25-4347-8780-49c69e46565a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:47.797143 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23495eaf-ab25-4347-8780-49c69e46565a admin admin] PATH_INFO: `/v3/projects/3413dab8ad594356ad8cde1976e33954/users/fea8c7b7ac1a486082ec87abad9a9845/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:47.805843 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 841/1717] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:47 2022] PUT /identity/v3/projects/e2c4bd2ed1ce46fe8bc9ed3e4a565f53/users/5267052717f441ebb8fc063f4ad4a26f/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:47.818133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59c875-6ed2-4d18-babb-c1fca999a249 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:47.818431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59c875-6ed2-4d18-babb-c1fca999a249 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:47.818714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59c875-6ed2-4d18-babb-c1fca999a249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:47.883617 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23495eaf-ab25-4347-8780-49c69e46565a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3413dab8ad594356ad8cde1976e33954, user_id=fea8c7b7ac1a486082ec87abad9a9845, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:47.899996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23495eaf-ab25-4347-8780-49c69e46565a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:47.907333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-0d59c875-6ed2-4d18-babb-c1fca999a249 None None] MFA Rules not processed for user `5267052717f441ebb8fc063f4ad4a26f`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:47.914633 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-0d59c875-6ed2-4d18-babb-c1fca999a249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:47.992493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 877/1718] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:47 2022] PUT /identity/v3/projects/3413dab8ad594356ad8cde1976e33954/users/fea8c7b7ac1a486082ec87abad9a9845/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:47.994935 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7cd66-bd76-43a6-ab00-81bca54876c2 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:47.995300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7cd66-bd76-43a6-ab00-81bca54876c2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:47.995561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7cd66-bd76-43a6-ab00-81bca54876c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:48.073332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4df7cd66-bd76-43a6-ab00-81bca54876c2 None None] MFA Rules not processed for user `fea8c7b7ac1a486082ec87abad9a9845`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:48.075561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4df7cd66-bd76-43a6-ab00-81bca54876c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:48.255910 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 842/1719] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:48.267522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59c875-6ed2-4d18-babb-c1fca999a249 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:48.270029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-0d59c875-6ed2-4d18-babb-c1fca999a249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:48.296209 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 878/1720] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:48.298402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7cd66-bd76-43a6-ab00-81bca54876c2 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:48.475293 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7cd66-bd76-43a6-ab00-81bca54876c2 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:48.481908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461e8e6-168c-4502-ad9d-03f81001e6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:48.483861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461e8e6-168c-4502-ad9d-03f81001e6da admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:48.484158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461e8e6-168c-4502-ad9d-03f81001e6da admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:48.485679 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461e8e6-168c-4502-ad9d-03f81001e6da admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:48.488606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461e8e6-168c-4502-ad9d-03f81001e6da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:48.496017 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461e8e6-168c-4502-ad9d-03f81001e6da admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:48.518560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59c875-6ed2-4d18-babb-c1fca999a249 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:48.586820 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 879/1721] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:48 2022] POST /identity/v3/projects => generated 365 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:48.614148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461e8e6-168c-4502-ad9d-03f81001e6da admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:48.719696 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1faee-961b-46b3-b4b6-f124e937509a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:48.720885 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1faee-961b-46b3-b4b6-f124e937509a service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:48.720885 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1faee-961b-46b3-b4b6-f124e937509a service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:48.720885 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1faee-961b-46b3-b4b6-f124e937509a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:48.847794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461e8e6-168c-4502-ad9d-03f81001e6da admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:48.869271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632fdd1-0d2b-4879-b285-bbabccc2a3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:48.870315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632fdd1-0d2b-4879-b285-bbabccc2a3ec admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:48.870769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632fdd1-0d2b-4879-b285-bbabccc2a3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:48.871251 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632fdd1-0d2b-4879-b285-bbabccc2a3ec admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:48.872101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632fdd1-0d2b-4879-b285-bbabccc2a3ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:48.874915 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632fdd1-0d2b-4879-b285-bbabccc2a3ec admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:48.911031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1faee-961b-46b3-b4b6-f124e937509a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:48.916598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1faee-961b-46b3-b4b6-f124e937509a service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:48.956693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 880/1722] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:48 2022] POST /identity/v3/users => generated 289 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:48.958926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632fdd1-0d2b-4879-b285-bbabccc2a3ec admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:48.966260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7632fdd1-0d2b-4879-b285-bbabccc2a3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:48.970369 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 843/1723] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:48.971625 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1faee-961b-46b3-b4b6-f124e937509a service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:49.010338 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632fdd1-0d2b-4879-b285-bbabccc2a3ec admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:49.022352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1faee-961b-46b3-b4b6-f124e937509a service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:49.030065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d353b-a92d-492d-bc94-5a1682393c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:49.031040 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d353b-a92d-492d-bc94-5a1682393c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:49.031508 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d353b-a92d-492d-bc94-5a1682393c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:49.031798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d353b-a92d-492d-bc94-5a1682393c37 admin admin] PATH_INFO: `/v3/projects/74dc3d60f0bc4ceb85d32720240c4032` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:49.034573 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d353b-a92d-492d-bc94-5a1682393c37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74dc3d60f0bc4ceb85d32720240c4032)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:49.036526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d353b-a92d-492d-bc94-5a1682393c37 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:49.043806 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ea913-70be-49d1-9b75-f421c98eaa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:49.044507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ea913-70be-49d1-9b75-f421c98eaa8b admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:49.044791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ea913-70be-49d1-9b75-f421c98eaa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:49.045067 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ea913-70be-49d1-9b75-f421c98eaa8b admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:49.045850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ea913-70be-49d1-9b75-f421c98eaa8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:49.047467 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ea913-70be-49d1-9b75-f421c98eaa8b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:49.067654 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 844/1724] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:49.071155 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ea913-70be-49d1-9b75-f421c98eaa8b admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:49.111598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ea913-70be-49d1-9b75-f421c98eaa8b admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:49.120479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6b4d8-12d9-4cb5-bde5-afb52e97510d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:49.121651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6b4d8-12d9-4cb5-bde5-afb52e97510d admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:49.122873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6b4d8-12d9-4cb5-bde5-afb52e97510d admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:49.123419 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6b4d8-12d9-4cb5-bde5-afb52e97510d admin admin] PATH_INFO: `/v3/projects/0755853c51354cf7aa942f6bdde3c948/users/9d868a7682d2464c8b163ae311dc13e9/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:49.160845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6b4d8-12d9-4cb5-bde5-afb52e97510d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0755853c51354cf7aa942f6bdde3c948, user_id=9d868a7682d2464c8b163ae311dc13e9, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:49.164909 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6b4d8-12d9-4cb5-bde5-afb52e97510d admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:49.215336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d88d353b-a92d-492d-bc94-5a1682393c37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74dc3d60f0bc4ceb85d32720240c4032', 'request_id': 'req-d88d353b-a92d-492d-bc94-5a1682393c37'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:49.222958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 845/1725] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:49 2022] PUT /identity/v3/projects/0755853c51354cf7aa942f6bdde3c948/users/9d868a7682d2464c8b163ae311dc13e9/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:29:49.227804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6b4d8-12d9-4cb5-bde5-afb52e97510d admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:49.240740 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d88d353b-a92d-492d-bc94-5a1682393c37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74dc3d60f0bc4ceb85d32720240c4032', 'request_id': 'req-d88d353b-a92d-492d-bc94-5a1682393c37'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:49.269649 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d88d353b-a92d-492d-bc94-5a1682393c37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74dc3d60f0bc4ceb85d32720240c4032', 'request_id': 'req-d88d353b-a92d-492d-bc94-5a1682393c37'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:49.298334 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d88d353b-a92d-492d-bc94-5a1682393c37 admin admin] The token cache is being invalidate because project 74dc3d60f0bc4ceb85d32720240c4032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:29:49.298616 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d88d353b-a92d-492d-bc94-5a1682393c37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:49.303696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d88d353b-a92d-492d-bc94-5a1682393c37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:49.305754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 881/1726] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:29:48 2022] DELETE /identity/v3/projects/74dc3d60f0bc4ceb85d32720240c4032 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:49.310882 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b5d1e-6e01-4528-b823-e5ff55d8ca92 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:49.311220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b5d1e-6e01-4528-b823-e5ff55d8ca92 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:49.312073 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b5d1e-6e01-4528-b823-e5ff55d8ca92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:49.373598 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-3d1b5d1e-6e01-4528-b823-e5ff55d8ca92 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:49.374881 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3d1b5d1e-6e01-4528-b823-e5ff55d8ca92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:49.491859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6b4d8-12d9-4cb5-bde5-afb52e97510d admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:49.523036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f768235-e9c1-4196-97d2-9aa3468c732c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:49.523773 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f768235-e9c1-4196-97d2-9aa3468c732c admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:49.524060 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f768235-e9c1-4196-97d2-9aa3468c732c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:49.524328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f768235-e9c1-4196-97d2-9aa3468c732c admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:49.525059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f768235-e9c1-4196-97d2-9aa3468c732c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:49.526653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f768235-e9c1-4196-97d2-9aa3468c732c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:49.569585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 846/1727] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:49.570327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f768235-e9c1-4196-97d2-9aa3468c732c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:49.591362 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1f768235-e9c1-4196-97d2-9aa3468c732c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:49.633699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 882/1728] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:29:49.634829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b5d1e-6e01-4528-b823-e5ff55d8ca92 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:49.637589 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3d1b5d1e-6e01-4528-b823-e5ff55d8ca92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:49.642138 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f768235-e9c1-4196-97d2-9aa3468c732c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:49.657572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ca3c5-cc29-4e9a-b851-76960bd4453c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:49.658476 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ca3c5-cc29-4e9a-b851-76960bd4453c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:49.658760 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ca3c5-cc29-4e9a-b851-76960bd4453c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:49.660016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ca3c5-cc29-4e9a-b851-76960bd4453c admin admin] PATH_INFO: `/v3/users/424bba92b5554defaa145a078f762cf0` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:49.665643 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b5d1e-6e01-4528-b823-e5ff55d8ca92 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:49.667365 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ca3c5-cc29-4e9a-b851-76960bd4453c admin admin] RBAC: Authorizing `identity:delete_user(user_id=424bba92b5554defaa145a078f762cf0)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:49.684343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ca3c5-cc29-4e9a-b851-76960bd4453c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:49.687930 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b100c8-4034-4913-98f5-c44702c27825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:49.705749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b100c8-4034-4913-98f5-c44702c27825 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:49.706044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b100c8-4034-4913-98f5-c44702c27825 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:49.706326 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b100c8-4034-4913-98f5-c44702c27825 admin admin] PATH_INFO: `/v3/projects/0755853c51354cf7aa942f6bdde3c948/users/9d868a7682d2464c8b163ae311dc13e9/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:49.719972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b100c8-4034-4913-98f5-c44702c27825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0755853c51354cf7aa942f6bdde3c948, user_id=9d868a7682d2464c8b163ae311dc13e9, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:49.722893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b100c8-4034-4913-98f5-c44702c27825 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:49.790692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 883/1729] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:49 2022] PUT /identity/v3/projects/0755853c51354cf7aa942f6bdde3c948/users/9d868a7682d2464c8b163ae311dc13e9/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:49.792137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baff48-a6d8-41ab-85b0-d407cdfabf01 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:49.792423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baff48-a6d8-41ab-85b0-d407cdfabf01 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:49.792700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baff48-a6d8-41ab-85b0-d407cdfabf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:49.892606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-83baff48-a6d8-41ab-85b0-d407cdfabf01 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:49.895567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-83baff48-a6d8-41ab-85b0-d407cdfabf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:49.901476 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-075ca3c5-cc29-4e9a-b851-76960bd4453c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '424bba92b5554defaa145a078f762cf0'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:49.924951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-075ca3c5-cc29-4e9a-b851-76960bd4453c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '424bba92b5554defaa145a078f762cf0'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:49.927077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-075ca3c5-cc29-4e9a-b851-76960bd4453c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '424bba92b5554defaa145a078f762cf0'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:49.941939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-075ca3c5-cc29-4e9a-b851-76960bd4453c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '424bba92b5554defaa145a078f762cf0'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:49.942502 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-075ca3c5-cc29-4e9a-b851-76960bd4453c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '424bba92b5554defaa145a078f762cf0'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:49.976609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 847/1730] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:29:49 2022] DELETE /identity/v3/users/424bba92b5554defaa145a078f762cf0 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:49.977950 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ae147-c689-4ec8-95d2-6a7594e30065 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:49.978235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ae147-c689-4ec8-95d2-6a7594e30065 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:49.978595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ae147-c689-4ec8-95d2-6a7594e30065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:50.074521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ff2ae147-c689-4ec8-95d2-6a7594e30065 None None] MFA Rules not processed for user `9d868a7682d2464c8b163ae311dc13e9`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:50.075663 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ff2ae147-c689-4ec8-95d2-6a7594e30065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:50.076183 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 884/1731] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:50.077588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3963cf-3a5d-4d2c-a083-2f70db5fa5d6 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:50.077889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3963cf-3a5d-4d2c-a083-2f70db5fa5d6 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:50.078147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3963cf-3a5d-4d2c-a083-2f70db5fa5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:50.170178 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-bb3963cf-3a5d-4d2c-a083-2f70db5fa5d6 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:50.172595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-bb3963cf-3a5d-4d2c-a083-2f70db5fa5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:50.322803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 885/1732] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:50.331207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3963cf-3a5d-4d2c-a083-2f70db5fa5d6 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:50.351052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 848/1733] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:50.363299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ae147-c689-4ec8-95d2-6a7594e30065 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:50.364602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3963cf-3a5d-4d2c-a083-2f70db5fa5d6 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:50.367061 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ff2ae147-c689-4ec8-95d2-6a7594e30065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:50.383633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f402b-6642-4b81-aefe-f9025792aac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:50.384323 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f402b-6642-4b81-aefe-f9025792aac2 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:50.384610 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f402b-6642-4b81-aefe-f9025792aac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:50.384888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f402b-6642-4b81-aefe-f9025792aac2 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:50.385620 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f402b-6642-4b81-aefe-f9025792aac2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:29:50.386100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f402b-6642-4b81-aefe-f9025792aac2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:50.387907 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f402b-6642-4b81-aefe-f9025792aac2 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:50.427423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 886/1734] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:29:50.428194 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f402b-6642-4b81-aefe-f9025792aac2 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:50.435889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ca5f402b-6642-4b81-aefe-f9025792aac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:50.501764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ae147-c689-4ec8-95d2-6a7594e30065 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:50.506028 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c7141-97a4-40b7-870d-af7c152fc10d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:50.506879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c7141-97a4-40b7-870d-af7c152fc10d service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:50.507360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c7141-97a4-40b7-870d-af7c152fc10d service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:50.507782 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c7141-97a4-40b7-870d-af7c152fc10d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:50.551641 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7c7141-97a4-40b7-870d-af7c152fc10d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:50.556266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7c7141-97a4-40b7-870d-af7c152fc10d service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:50.572599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f402b-6642-4b81-aefe-f9025792aac2 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:50.581658 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 849/1735] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:50.581882 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1efe6-a367-4435-9b7d-8220947172c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:50.582621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1efe6-a367-4435-9b7d-8220947172c3 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:50.582982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1efe6-a367-4435-9b7d-8220947172c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:50.583366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1efe6-a367-4435-9b7d-8220947172c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:50.583876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c7141-97a4-40b7-870d-af7c152fc10d service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:50.605319 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1efe6-a367-4435-9b7d-8220947172c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:50.606884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c7141-97a4-40b7-870d-af7c152fc10d service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:50.608851 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1efe6-a367-4435-9b7d-8220947172c3 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:50.613424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1ece7-44ef-46d7-bff8-7c916d5831c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:50.614759 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1ece7-44ef-46d7-bff8-7c916d5831c2 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:50.615411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1ece7-44ef-46d7-bff8-7c916d5831c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:50.615830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1ece7-44ef-46d7-bff8-7c916d5831c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:50.616862 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1ece7-44ef-46d7-bff8-7c916d5831c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:50.635794 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1ece7-44ef-46d7-bff8-7c916d5831c2 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:50.651266 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 887/1736] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:50 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:50.725844 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1efe6-a367-4435-9b7d-8220947172c3 service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:50.727672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-faf1efe6-a367-4435-9b7d-8220947172c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:50.736114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 850/1737] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:50 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:50.736720 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e190eed-afef-4170-8aed-81d52dfd05a4 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:50.742052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e190eed-afef-4170-8aed-81d52dfd05a4 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:50.742907 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e190eed-afef-4170-8aed-81d52dfd05a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:50.767597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-2e190eed-afef-4170-8aed-81d52dfd05a4 None None] MFA Rules not processed for user `6b2b6318776b422d9701351ba6b4ee27`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:50.768800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-2e190eed-afef-4170-8aed-81d52dfd05a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:50.772366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1efe6-a367-4435-9b7d-8220947172c3 service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:50.881904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 851/1738] 10.222.0.36 () {58 vars in 1015 bytes} [Mon May 9 18:29:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:50.882935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e190eed-afef-4170-8aed-81d52dfd05a4 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:50.948909 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d31d05-acc7-4a80-9b7d-8d32d5083499 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:50.950116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d31d05-acc7-4a80-9b7d-8d32d5083499 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:50.959957 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d31d05-acc7-4a80-9b7d-8d32d5083499 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:50.960520 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d31d05-acc7-4a80-9b7d-8d32d5083499 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:50.966464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-17d31d05-acc7-4a80-9b7d-8d32d5083499 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:51.110289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d31d05-acc7-4a80-9b7d-8d32d5083499 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:51.114329 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d31d05-acc7-4a80-9b7d-8d32d5083499 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:51.136746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e190eed-afef-4170-8aed-81d52dfd05a4 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:51.160561 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae36190-8dc6-45bd-8ed2-603727825dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:51.161279 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae36190-8dc6-45bd-8ed2-603727825dfa admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:51.161557 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae36190-8dc6-45bd-8ed2-603727825dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:51.161832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae36190-8dc6-45bd-8ed2-603727825dfa admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:51.162530 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae36190-8dc6-45bd-8ed2-603727825dfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:51.164499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae36190-8dc6-45bd-8ed2-603727825dfa admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:51.209172 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 888/1739] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:50 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:29:51.212568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 852/1740] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:50 2022] POST /identity/v3/users => generated 284 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:51.213629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae36190-8dc6-45bd-8ed2-603727825dfa admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:51.219010 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-dae36190-8dc6-45bd-8ed2-603727825dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:51.229974 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d31d05-acc7-4a80-9b7d-8d32d5083499 service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:51.247259 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae36190-8dc6-45bd-8ed2-603727825dfa admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:51.257673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d82992-5b7a-4d0b-99ac-c7f9a7cf70b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:51.258593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d82992-5b7a-4d0b-99ac-c7f9a7cf70b6 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:51.259033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d82992-5b7a-4d0b-99ac-c7f9a7cf70b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:51.259489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d82992-5b7a-4d0b-99ac-c7f9a7cf70b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:51.284240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d82992-5b7a-4d0b-99ac-c7f9a7cf70b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:51.285586 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d31d05-acc7-4a80-9b7d-8d32d5083499 service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:51.286175 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d82992-5b7a-4d0b-99ac-c7f9a7cf70b6 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:51.298710 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39992b50-34a6-4bfa-90ad-b41b1d851d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:51.300497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39992b50-34a6-4bfa-90ad-b41b1d851d9f admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:51.302384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39992b50-34a6-4bfa-90ad-b41b1d851d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:51.304878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39992b50-34a6-4bfa-90ad-b41b1d851d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:51.305672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39992b50-34a6-4bfa-90ad-b41b1d851d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:51.308905 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39992b50-34a6-4bfa-90ad-b41b1d851d9f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:51.313531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 853/1741] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:51.321634 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d82992-5b7a-4d0b-99ac-c7f9a7cf70b6 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:51.323325 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f7d82992-5b7a-4d0b-99ac-c7f9a7cf70b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:51.337200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 889/1742] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:51.343295 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39992b50-34a6-4bfa-90ad-b41b1d851d9f admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:51.353377 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d82992-5b7a-4d0b-99ac-c7f9a7cf70b6 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:51.364446 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39992b50-34a6-4bfa-90ad-b41b1d851d9f admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:51.368717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:51.369850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:51.370316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:51.370760 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] PATH_INFO: `/v3/projects/a0215e7469b54a89bb356d03a8c87aae` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:51.372158 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0215e7469b54a89bb356d03a8c87aae)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:51.375894 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283775c-47c5-4694-a061-3f589a79ab05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:51.376074 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:51.376610 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283775c-47c5-4694-a061-3f589a79ab05 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:51.376902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283775c-47c5-4694-a061-3f589a79ab05 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:51.377186 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283775c-47c5-4694-a061-3f589a79ab05 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:51.378173 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283775c-47c5-4694-a061-3f589a79ab05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:51.382172 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283775c-47c5-4694-a061-3f589a79ab05 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:51.411459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 890/1743] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:51.412462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283775c-47c5-4694-a061-3f589a79ab05 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:51.441483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283775c-47c5-4694-a061-3f589a79ab05 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:51.460107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50aac3a-70b1-4d36-850a-2580d3993660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:51.462047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50aac3a-70b1-4d36-850a-2580d3993660 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:51.472412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50aac3a-70b1-4d36-850a-2580d3993660 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:51.472749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50aac3a-70b1-4d36-850a-2580d3993660 admin admin] PATH_INFO: `/v3/projects/36bd9f2168e24ceda5308fe4919649be/users/1ac317b9a0ea4aca84c20c39c5896a89/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:51.486205 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0215e7469b54a89bb356d03a8c87aae', 'request_id': 'req-1498f8bb-82f3-4d79-9c92-f491aaa8da25'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:51.501012 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0215e7469b54a89bb356d03a8c87aae', 'request_id': 'req-1498f8bb-82f3-4d79-9c92-f491aaa8da25'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:51.515806 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0215e7469b54a89bb356d03a8c87aae', 'request_id': 'req-1498f8bb-82f3-4d79-9c92-f491aaa8da25'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:51.536504 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50aac3a-70b1-4d36-850a-2580d3993660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36bd9f2168e24ceda5308fe4919649be, user_id=1ac317b9a0ea4aca84c20c39c5896a89, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:51.540589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] The token cache is being invalidate because project a0215e7469b54a89bb356d03a8c87aae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:29:51.540877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:51.544132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:51.548676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 854/1744] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:29:51 2022] DELETE /identity/v3/projects/a0215e7469b54a89bb356d03a8c87aae => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:29:51.552644 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:51.554044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:51.554986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50aac3a-70b1-4d36-850a-2580d3993660 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:51.622608 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 891/1745] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:51 2022] PUT /identity/v3/projects/36bd9f2168e24ceda5308fe4919649be/users/1ac317b9a0ea4aca84c20c39c5896a89/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:51.624972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5378ec9-810c-46c6-8b49-d7a84a5f581e None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:51.625265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5378ec9-810c-46c6-8b49-d7a84a5f581e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:51.625548 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5378ec9-810c-46c6-8b49-d7a84a5f581e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:51.742981 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b5378ec9-810c-46c6-8b49-d7a84a5f581e None None] MFA Rules not processed for user `1ac317b9a0ea4aca84c20c39c5896a89`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:51.744711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498f8bb-82f3-4d79-9c92-f491aaa8da25 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:51.745796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b5378ec9-810c-46c6-8b49-d7a84a5f581e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:51.948503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54094918-cc25-4697-a7c8-32b619acc743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:51.949423 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54094918-cc25-4697-a7c8-32b619acc743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:51.949713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54094918-cc25-4697-a7c8-32b619acc743 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:51.952620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54094918-cc25-4697-a7c8-32b619acc743 admin admin] PATH_INFO: `/v3/users/ac9e6b1aa85a4ec8a0bf08cdab75082d` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:51.956334 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54094918-cc25-4697-a7c8-32b619acc743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac9e6b1aa85a4ec8a0bf08cdab75082d)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:51.958286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54094918-cc25-4697-a7c8-32b619acc743 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:52.054415 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 892/1746] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:52.061983 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5378ec9-810c-46c6-8b49-d7a84a5f581e None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:52.065032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b5378ec9-810c-46c6-8b49-d7a84a5f581e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:52.081803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-54094918-cc25-4697-a7c8-32b619acc743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac9e6b1aa85a4ec8a0bf08cdab75082d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:52.101078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-54094918-cc25-4697-a7c8-32b619acc743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac9e6b1aa85a4ec8a0bf08cdab75082d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:52.102833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-54094918-cc25-4697-a7c8-32b619acc743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac9e6b1aa85a4ec8a0bf08cdab75082d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:52.118387 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-54094918-cc25-4697-a7c8-32b619acc743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac9e6b1aa85a4ec8a0bf08cdab75082d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:52.119078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-54094918-cc25-4697-a7c8-32b619acc743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac9e6b1aa85a4ec8a0bf08cdab75082d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:52.164619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 855/1747] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:29:51 2022] DELETE /identity/v3/users/ac9e6b1aa85a4ec8a0bf08cdab75082d => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:52.192041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5378ec9-810c-46c6-8b49-d7a84a5f581e None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:52.377680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d660b-82f8-4d6c-8eeb-0916a310b691 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:52.378362 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d660b-82f8-4d6c-8eeb-0916a310b691 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:52.378647 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d660b-82f8-4d6c-8eeb-0916a310b691 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:52.378926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d660b-82f8-4d6c-8eeb-0916a310b691 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:52.384016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b80d660b-82f8-4d6c-8eeb-0916a310b691 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:52.595189 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d660b-82f8-4d6c-8eeb-0916a310b691 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:52.601038 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d660b-82f8-4d6c-8eeb-0916a310b691 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:52.636577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 893/1748] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:52.894233 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb387590-65d4-4ba3-9f64-919ac57f925d None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:52.894712 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb387590-65d4-4ba3-9f64-919ac57f925d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:52.895129 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb387590-65d4-4ba3-9f64-919ac57f925d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:52.930863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d660b-82f8-4d6c-8eeb-0916a310b691 service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:52.933886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b80d660b-82f8-4d6c-8eeb-0916a310b691 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:52.964691 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-cb387590-65d4-4ba3-9f64-919ac57f925d None None] MFA Rules not processed for user `90fdc96d6e6649a498bb3a8659fcc030`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:52.965890 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-cb387590-65d4-4ba3-9f64-919ac57f925d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:53.095069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d660b-82f8-4d6c-8eeb-0916a310b691 service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:53.120392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:53.121282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:53.121565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:53.121837 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] PATH_INFO: `/v3/projects/2596302735dd4b07b3d9ca4de4591ed6` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:53.128264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2596302735dd4b07b3d9ca4de4591ed6)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:53.130184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:53.201314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 856/1749] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:53.219056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb387590-65d4-4ba3-9f64-919ac57f925d None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:53.221040 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-cb387590-65d4-4ba3-9f64-919ac57f925d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:53.292708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2596302735dd4b07b3d9ca4de4591ed6', 'request_id': 'req-50aa50b9-6606-4e2c-8adf-1ce01df270d8'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:53.326439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2596302735dd4b07b3d9ca4de4591ed6', 'request_id': 'req-50aa50b9-6606-4e2c-8adf-1ce01df270d8'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:53.349432 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2596302735dd4b07b3d9ca4de4591ed6', 'request_id': 'req-50aa50b9-6606-4e2c-8adf-1ce01df270d8'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:53.403086 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] The token cache is being invalidate because project 2596302735dd4b07b3d9ca4de4591ed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:29:53.407216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:53.408096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:53.410096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 894/1750] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:29:52 2022] DELETE /identity/v3/projects/2596302735dd4b07b3d9ca4de4591ed6 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:53.418701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:53.420451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:53.474186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb387590-65d4-4ba3-9f64-919ac57f925d None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:53.670596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa50b9-6606-4e2c-8adf-1ce01df270d8 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:53.676660 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92ff7b-36ce-49ce-8247-4d2cf4a5ff20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:53.679986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92ff7b-36ce-49ce-8247-4d2cf4a5ff20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:53.680454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92ff7b-36ce-49ce-8247-4d2cf4a5ff20 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:53.681034 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92ff7b-36ce-49ce-8247-4d2cf4a5ff20 admin admin] PATH_INFO: `/v3/users/5267052717f441ebb8fc063f4ad4a26f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:53.682656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92ff7b-36ce-49ce-8247-4d2cf4a5ff20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5267052717f441ebb8fc063f4ad4a26f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:53.685059 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92ff7b-36ce-49ce-8247-4d2cf4a5ff20 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:53.728099 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc746d3-de9e-43f7-b5b0-eaa034f2bbc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:53.728881 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc746d3-de9e-43f7-b5b0-eaa034f2bbc1 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:53.729234 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc746d3-de9e-43f7-b5b0-eaa034f2bbc1 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:53.729580 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc746d3-de9e-43f7-b5b0-eaa034f2bbc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:53.733900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9fc746d3-de9e-43f7-b5b0-eaa034f2bbc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:53.764207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2d92ff7b-36ce-49ce-8247-4d2cf4a5ff20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5267052717f441ebb8fc063f4ad4a26f'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:53.765746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2d92ff7b-36ce-49ce-8247-4d2cf4a5ff20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5267052717f441ebb8fc063f4ad4a26f'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:53.782348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2d92ff7b-36ce-49ce-8247-4d2cf4a5ff20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5267052717f441ebb8fc063f4ad4a26f'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:53.799240 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2d92ff7b-36ce-49ce-8247-4d2cf4a5ff20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5267052717f441ebb8fc063f4ad4a26f'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:53.800242 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2d92ff7b-36ce-49ce-8247-4d2cf4a5ff20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5267052717f441ebb8fc063f4ad4a26f'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:53.846937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 895/1751] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:29:53 2022] DELETE /identity/v3/users/5267052717f441ebb8fc063f4ad4a26f => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:53.947671 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc746d3-de9e-43f7-b5b0-eaa034f2bbc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:53.950092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc746d3-de9e-43f7-b5b0-eaa034f2bbc1 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:53.973439 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 857/1752] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 755 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:29:55.201772 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92ff7b-36ce-49ce-8247-4d2cf4a5ff20 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:55.204039 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2d92ff7b-36ce-49ce-8247-4d2cf4a5ff20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:55.464045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92ff7b-36ce-49ce-8247-4d2cf4a5ff20 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:55.488304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b3285-22c3-4c70-b841-e13383aa9c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:55.489443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b3285-22c3-4c70-b841-e13383aa9c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:55.489875 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b3285-22c3-4c70-b841-e13383aa9c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:55.490322 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b3285-22c3-4c70-b841-e13383aa9c25 admin admin] PATH_INFO: `/v3/projects/e2c4bd2ed1ce46fe8bc9ed3e4a565f53` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:55.494975 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b3285-22c3-4c70-b841-e13383aa9c25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2c4bd2ed1ce46fe8bc9ed3e4a565f53)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:55.496965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b3285-22c3-4c70-b841-e13383aa9c25 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:55.626512 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e22b3285-22c3-4c70-b841-e13383aa9c25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2c4bd2ed1ce46fe8bc9ed3e4a565f53', 'request_id': 'req-e22b3285-22c3-4c70-b841-e13383aa9c25'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:55.634040 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e22b3285-22c3-4c70-b841-e13383aa9c25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2c4bd2ed1ce46fe8bc9ed3e4a565f53', 'request_id': 'req-e22b3285-22c3-4c70-b841-e13383aa9c25'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:55.644225 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e22b3285-22c3-4c70-b841-e13383aa9c25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2c4bd2ed1ce46fe8bc9ed3e4a565f53', 'request_id': 'req-e22b3285-22c3-4c70-b841-e13383aa9c25'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:55.682581 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e22b3285-22c3-4c70-b841-e13383aa9c25 admin admin] The token cache is being invalidate because project e2c4bd2ed1ce46fe8bc9ed3e4a565f53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:29:55.683152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e22b3285-22c3-4c70-b841-e13383aa9c25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:55.684122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e22b3285-22c3-4c70-b841-e13383aa9c25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:55.688706 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 896/1753] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:29:55 2022] DELETE /identity/v3/projects/e2c4bd2ed1ce46fe8bc9ed3e4a565f53 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:55.693545 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25242e35-d69f-4199-a7b9-c569a7b1dcfb None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:55.694135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25242e35-d69f-4199-a7b9-c569a7b1dcfb None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:55.694135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25242e35-d69f-4199-a7b9-c569a7b1dcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:55.729461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-25242e35-d69f-4199-a7b9-c569a7b1dcfb None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:55.730728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-25242e35-d69f-4199-a7b9-c569a7b1dcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:55.960663 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 858/1754] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:55.968818 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc9387-df3e-4819-9dd4-35c8672ce6a4 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:55.969677 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc9387-df3e-4819-9dd4-35c8672ce6a4 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:55.970091 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc9387-df3e-4819-9dd4-35c8672ce6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:56.015435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-1cbc9387-df3e-4819-9dd4-35c8672ce6a4 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:56.052187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-1cbc9387-df3e-4819-9dd4-35c8672ce6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:56.085317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 897/1755] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:56.089252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25242e35-d69f-4199-a7b9-c569a7b1dcfb None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:56.105808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25242e35-d69f-4199-a7b9-c569a7b1dcfb None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:56.119310 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbefb7-1db7-4761-9bdc-fb4eb09e164f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:56.120270 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbefb7-1db7-4761-9bdc-fb4eb09e164f admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:56.120703 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbefb7-1db7-4761-9bdc-fb4eb09e164f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:56.121137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbefb7-1db7-4761-9bdc-fb4eb09e164f admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:56.122116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbefb7-1db7-4761-9bdc-fb4eb09e164f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:29:56.122620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbefb7-1db7-4761-9bdc-fb4eb09e164f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:56.131979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbefb7-1db7-4761-9bdc-fb4eb09e164f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:56.161199 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc9387-df3e-4819-9dd4-35c8672ce6a4 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:56.172131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc9387-df3e-4819-9dd4-35c8672ce6a4 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:56.176121 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 859/1756] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:29:56.192461 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a089c2-fcaf-4578-a99f-ffd6cf874ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:56.193702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a089c2-fcaf-4578-a99f-ffd6cf874ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:56.193702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a089c2-fcaf-4578-a99f-ffd6cf874ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:56.193702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a089c2-fcaf-4578-a99f-ffd6cf874ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:56.194418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a089c2-fcaf-4578-a99f-ffd6cf874ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:56.202601 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a089c2-fcaf-4578-a99f-ffd6cf874ea7 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:56.300323 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 898/1757] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:56 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:56.304968 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbefb7-1db7-4761-9bdc-fb4eb09e164f admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:56.341464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbefb7-1db7-4761-9bdc-fb4eb09e164f admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:56.648429 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13631ac6-6d6c-488a-83a6-6e66cbdfcada None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:56.648711 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13631ac6-6d6c-488a-83a6-6e66cbdfcada None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:56.648992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13631ac6-6d6c-488a-83a6-6e66cbdfcada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:56.721446 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-13631ac6-6d6c-488a-83a6-6e66cbdfcada None None] MFA Rules not processed for user `acae3ad19a7e45b9877520dff8f5b995`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:56.723336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-13631ac6-6d6c-488a-83a6-6e66cbdfcada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:56.746496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e62958-8608-422c-a01f-c8c651005ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:56.747370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e62958-8608-422c-a01f-c8c651005ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:56.747766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e62958-8608-422c-a01f-c8c651005ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:56.748203 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e62958-8608-422c-a01f-c8c651005ba3 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:56.749573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e62958-8608-422c-a01f-c8c651005ba3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:56.754269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e62958-8608-422c-a01f-c8c651005ba3 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:56.818757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 860/1758] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:56 2022] POST /identity/v3/users => generated 286 bytes in 514 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:29:56.823613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e62958-8608-422c-a01f-c8c651005ba3 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:56.856266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e62958-8608-422c-a01f-c8c651005ba3 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:56.880557 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66520fe-a3e7-4c2a-bb60-6758661cf804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:56.881857 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66520fe-a3e7-4c2a-bb60-6758661cf804 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:56.883454 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66520fe-a3e7-4c2a-bb60-6758661cf804 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:56.883984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66520fe-a3e7-4c2a-bb60-6758661cf804 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:56.885407 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66520fe-a3e7-4c2a-bb60-6758661cf804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:56.888310 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66520fe-a3e7-4c2a-bb60-6758661cf804 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:56.927482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 861/1759] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:29:56.930589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8d64-9b65-4465-b8d1-5ab3d4c683a5 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:56.931963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8d64-9b65-4465-b8d1-5ab3d4c683a5 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:56.934082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8d64-9b65-4465-b8d1-5ab3d4c683a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:56.977905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-fb0b8d64-9b65-4465-b8d1-5ab3d4c683a5 None None] MFA Rules not processed for user `f99208187e4e4d2eb9f52107ae8f50f2`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:56.980635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-fb0b8d64-9b65-4465-b8d1-5ab3d4c683a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:57.001146 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 899/1760] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:57.003496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13631ac6-6d6c-488a-83a6-6e66cbdfcada None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:57.051568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13631ac6-6d6c-488a-83a6-6e66cbdfcada None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:57.071792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2337625-508d-47a8-bbff-5a8e6712f0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:57.074500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2337625-508d-47a8-bbff-5a8e6712f0ee admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:57.077032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2337625-508d-47a8-bbff-5a8e6712f0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:57.080145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2337625-508d-47a8-bbff-5a8e6712f0ee admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:57.081115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2337625-508d-47a8-bbff-5a8e6712f0ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:57.089784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2337625-508d-47a8-bbff-5a8e6712f0ee admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:57.149177 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 900/1761] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:57.152584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b29d2-f843-403c-9a1c-b8df0df29a56 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:57.165610 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b29d2-f843-403c-9a1c-b8df0df29a56 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:57.166001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b29d2-f843-403c-9a1c-b8df0df29a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:57.199712 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 862/1762] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:57.200979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b8d64-9b65-4465-b8d1-5ab3d4c683a5 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:57.236288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b8d64-9b65-4465-b8d1-5ab3d4c683a5 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:57.254605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48a5bc-db72-479a-a69a-9e0f988124ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:57.258362 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48a5bc-db72-479a-a69a-9e0f988124ed admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:57.258815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48a5bc-db72-479a-a69a-9e0f988124ed admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:57.259104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48a5bc-db72-479a-a69a-9e0f988124ed admin admin] PATH_INFO: `/v3/projects/df9194d5f9174b7c943c71fa7bfb3318/users/c1d626990b114bf3bd19bd6386831bb5/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:57.272529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c23b29d2-f843-403c-9a1c-b8df0df29a56 None None] MFA Rules not processed for user `acae3ad19a7e45b9877520dff8f5b995`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:57.281864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c23b29d2-f843-403c-9a1c-b8df0df29a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:57.319477 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48a5bc-db72-479a-a69a-9e0f988124ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=df9194d5f9174b7c943c71fa7bfb3318, user_id=c1d626990b114bf3bd19bd6386831bb5, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:57.321714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 901/1763] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:57.324419 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48a5bc-db72-479a-a69a-9e0f988124ed admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:57.336822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b29d2-f843-403c-9a1c-b8df0df29a56 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:57.338687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c23b29d2-f843-403c-9a1c-b8df0df29a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:57.367710 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 863/1764] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:57 2022] PUT /identity/v3/projects/df9194d5f9174b7c943c71fa7bfb3318/users/c1d626990b114bf3bd19bd6386831bb5/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:57.377056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135ff24-5395-4341-80b9-ef916ee8e252 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:57.378602 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135ff24-5395-4341-80b9-ef916ee8e252 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:57.381422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135ff24-5395-4341-80b9-ef916ee8e252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:57.439415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f135ff24-5395-4341-80b9-ef916ee8e252 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:57.439415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f135ff24-5395-4341-80b9-ef916ee8e252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:57.590662 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b29d2-f843-403c-9a1c-b8df0df29a56 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:57.676051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 864/1765] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:57.677021 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135ff24-5395-4341-80b9-ef916ee8e252 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:57.680136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f135ff24-5395-4341-80b9-ef916ee8e252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:57.718719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135ff24-5395-4341-80b9-ef916ee8e252 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:57.730750 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b67f70-6b82-4adc-ad4d-864ecf5f0db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:57.732465 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b67f70-6b82-4adc-ad4d-864ecf5f0db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:57.732683 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b67f70-6b82-4adc-ad4d-864ecf5f0db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:57.733571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b67f70-6b82-4adc-ad4d-864ecf5f0db9 admin admin] PATH_INFO: `/v3/users/fea8c7b7ac1a486082ec87abad9a9845` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:57.736056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b67f70-6b82-4adc-ad4d-864ecf5f0db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea8c7b7ac1a486082ec87abad9a9845)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:57.739991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b67f70-6b82-4adc-ad4d-864ecf5f0db9 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:57.770817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e95a5-9beb-4d15-aa57-9d25c4ee1815 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:57.773247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e95a5-9beb-4d15-aa57-9d25c4ee1815 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:57.773811 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e95a5-9beb-4d15-aa57-9d25c4ee1815 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:57.780855 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e95a5-9beb-4d15-aa57-9d25c4ee1815 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:57.893547 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c0b67f70-6b82-4adc-ad4d-864ecf5f0db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea8c7b7ac1a486082ec87abad9a9845'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:57.923902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c0b67f70-6b82-4adc-ad4d-864ecf5f0db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea8c7b7ac1a486082ec87abad9a9845'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:57.926226 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c0b67f70-6b82-4adc-ad4d-864ecf5f0db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea8c7b7ac1a486082ec87abad9a9845'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:57.948586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c0b67f70-6b82-4adc-ad4d-864ecf5f0db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea8c7b7ac1a486082ec87abad9a9845'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:57.952789 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c0b67f70-6b82-4adc-ad4d-864ecf5f0db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea8c7b7ac1a486082ec87abad9a9845'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:57.988943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e95a5-9beb-4d15-aa57-9d25c4ee1815 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:57.990551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e95a5-9beb-4d15-aa57-9d25c4ee1815 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:58.005573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 865/1766] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:29:57 2022] DELETE /identity/v3/users/fea8c7b7ac1a486082ec87abad9a9845 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:58.015106 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5731c0-a53e-4dc1-b29c-3c56ad3ccb2a None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:58.015435 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5731c0-a53e-4dc1-b29c-3c56ad3ccb2a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:58.015723 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5731c0-a53e-4dc1-b29c-3c56ad3ccb2a None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:58.021348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 866/1767] 10.222.0.25 () {54 vars in 921 bytes} [Mon May 9 18:29:58 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:29:58.022223 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5731c0-a53e-4dc1-b29c-3c56ad3ccb2a None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:58.045904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 902/1768] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:58.047035 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9ae9b-22b2-4930-b364-62d36b873dfa None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:58.047356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9ae9b-22b2-4930-b364-62d36b873dfa None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:58.047636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9ae9b-22b2-4930-b364-62d36b873dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:58.049094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-0b5731c0-a53e-4dc1-b29c-3c56ad3ccb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:58.115224 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9ee9ae9b-22b2-4930-b364-62d36b873dfa None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:58.118687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9ee9ae9b-22b2-4930-b364-62d36b873dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:58.235309 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 903/1769] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:58.241812 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5f5b1-553d-4446-b752-a2eea4f9e2a2 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:58.242344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5f5b1-553d-4446-b752-a2eea4f9e2a2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:58.242802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5f5b1-553d-4446-b752-a2eea4f9e2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:58.268836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5731c0-a53e-4dc1-b29c-3c56ad3ccb2a None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:58.272908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb59712-2995-48d1-8f30-eb5e30dcb7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:58.273828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb59712-2995-48d1-8f30-eb5e30dcb7fe service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:58.274353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb59712-2995-48d1-8f30-eb5e30dcb7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:58.275009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb59712-2995-48d1-8f30-eb5e30dcb7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:58.275453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b7e5f5b1-553d-4446-b752-a2eea4f9e2a2 None None] MFA Rules not processed for user `f769ee9f31354e5296645fb1e3bd84b6`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:58.276685 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b7e5f5b1-553d-4446-b752-a2eea4f9e2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:58.278063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-5eb59712-2995-48d1-8f30-eb5e30dcb7fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:58.293523 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb59712-2995-48d1-8f30-eb5e30dcb7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:58.296226 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb59712-2995-48d1-8f30-eb5e30dcb7fe service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:58.317333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 867/1770] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:58.318557 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb59712-2995-48d1-8f30-eb5e30dcb7fe service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:58.321115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-5eb59712-2995-48d1-8f30-eb5e30dcb7fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:58.336033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb59712-2995-48d1-8f30-eb5e30dcb7fe service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:58.342094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:58.343531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:58.344362 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:58.350522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0 admin admin] PATH_INFO: `/v3/projects/099b98b265a34a7ca16ac33f9b9890ae` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:58.352614 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=099b98b265a34a7ca16ac33f9b9890ae)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:58.355071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:58.422641 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 904/1771] 10.222.0.25 () {58 vars in 1018 bytes} [Mon May 9 18:29:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:58.423605 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5f5b1-553d-4446-b752-a2eea4f9e2a2 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:58.432790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5f5b1-553d-4446-b752-a2eea4f9e2a2 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:58.456779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '099b98b265a34a7ca16ac33f9b9890ae', 'request_id': 'req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:58.466482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '099b98b265a34a7ca16ac33f9b9890ae', 'request_id': 'req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:58.481448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '099b98b265a34a7ca16ac33f9b9890ae', 'request_id': 'req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:58.501793 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0 admin admin] The token cache is being invalidate because project 099b98b265a34a7ca16ac33f9b9890ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:29:58.502002 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:58.503595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9a2a0aef-7d5e-4394-96d7-0d7dd7a0e8b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:29:58.505209 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 868/1772] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:29:58 2022] DELETE /identity/v3/projects/099b98b265a34a7ca16ac33f9b9890ae => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:29:58.512147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbde1c5-baa3-4060-ad60-96ae94e49939 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:58.512437 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbde1c5-baa3-4060-ad60-96ae94e49939 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:58.512892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbde1c5-baa3-4060-ad60-96ae94e49939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:58.534465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f57656-eedb-4dd0-92ee-9df0250833ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:58.535275 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f57656-eedb-4dd0-92ee-9df0250833ab service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:58.535631 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f57656-eedb-4dd0-92ee-9df0250833ab service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:58.535809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ecbde1c5-baa3-4060-ad60-96ae94e49939 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:58.536410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f57656-eedb-4dd0-92ee-9df0250833ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:58.537320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ecbde1c5-baa3-4060-ad60-96ae94e49939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:58.538676 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-42f57656-eedb-4dd0-92ee-9df0250833ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:58.687429 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f57656-eedb-4dd0-92ee-9df0250833ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:58.688884 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f57656-eedb-4dd0-92ee-9df0250833ab service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:58.711897 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 905/1773] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:29:58.741659 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 869/1774] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:58.756136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c8a62-e9d0-4535-a28e-e3f5f848fb9c None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:58.756136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c8a62-e9d0-4535-a28e-e3f5f848fb9c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:58.756136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c8a62-e9d0-4535-a28e-e3f5f848fb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:58.793155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2d5c8a62-e9d0-4535-a28e-e3f5f848fb9c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:58.794392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2d5c8a62-e9d0-4535-a28e-e3f5f848fb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:58.825242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbde1c5-baa3-4060-ad60-96ae94e49939 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:58.838456 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbde1c5-baa3-4060-ad60-96ae94e49939 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:58.839405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 906/1775] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:29:58.847571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50eac28-4c3b-402a-bc7e-48c3a62b462c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:58.848681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50eac28-4c3b-402a-bc7e-48c3a62b462c admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:58.849155 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50eac28-4c3b-402a-bc7e-48c3a62b462c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:58.849480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50eac28-4c3b-402a-bc7e-48c3a62b462c admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:58.851248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50eac28-4c3b-402a-bc7e-48c3a62b462c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:29:58.851248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50eac28-4c3b-402a-bc7e-48c3a62b462c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:58.859521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50eac28-4c3b-402a-bc7e-48c3a62b462c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:58.883992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c8a62-e9d0-4535-a28e-e3f5f848fb9c None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:58.889971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 870/1776] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:29:58.911840 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c8a62-e9d0-4535-a28e-e3f5f848fb9c None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:58.919829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318df97-e8b8-49f1-83dd-7462e431c333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:58.921065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318df97-e8b8-49f1-83dd-7462e431c333 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:58.921349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318df97-e8b8-49f1-83dd-7462e431c333 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:58.921623 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318df97-e8b8-49f1-83dd-7462e431c333 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:58.922654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318df97-e8b8-49f1-83dd-7462e431c333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:58.925379 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318df97-e8b8-49f1-83dd-7462e431c333 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:58.979957 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50eac28-4c3b-402a-bc7e-48c3a62b462c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:58.988673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 907/1777] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:58 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:58.993047 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50eac28-4c3b-402a-bc7e-48c3a62b462c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:59.079063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053cfef-5f83-4982-85b6-e5a86d7d69a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:59.079915 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053cfef-5f83-4982-85b6-e5a86d7d69a7 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:59.080202 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053cfef-5f83-4982-85b6-e5a86d7d69a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:59.080624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053cfef-5f83-4982-85b6-e5a86d7d69a7 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:59.081313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053cfef-5f83-4982-85b6-e5a86d7d69a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:59.083332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053cfef-5f83-4982-85b6-e5a86d7d69a7 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:59.130755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 871/1778] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:58 2022] POST /identity/v3/users => generated 290 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:59.134474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318df97-e8b8-49f1-83dd-7462e431c333 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:59.148569 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318df97-e8b8-49f1-83dd-7462e431c333 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:59.162103 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4e573-b672-446b-92be-0645f1e74364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:59.162982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4e573-b672-446b-92be-0645f1e74364 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:59.163304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4e573-b672-446b-92be-0645f1e74364 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:59.163586 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4e573-b672-446b-92be-0645f1e74364 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:59.167303 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4e573-b672-446b-92be-0645f1e74364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:59.171291 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4e573-b672-446b-92be-0645f1e74364 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:59.202895 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 908/1779] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:59.203979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053cfef-5f83-4982-85b6-e5a86d7d69a7 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:59.220761 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053cfef-5f83-4982-85b6-e5a86d7d69a7 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:59.236946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae300251-8645-448a-8213-d5d21e784590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:59.237652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae300251-8645-448a-8213-d5d21e784590 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:59.237942 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae300251-8645-448a-8213-d5d21e784590 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:59.238216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae300251-8645-448a-8213-d5d21e784590 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:59.244044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae300251-8645-448a-8213-d5d21e784590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:59.248123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae300251-8645-448a-8213-d5d21e784590 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:59.288879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4e573-b672-446b-92be-0645f1e74364 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:59.299444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 872/1780] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:29:59.302503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4e573-b672-446b-92be-0645f1e74364 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:59.317928 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4e6e0-3a72-4935-9f14-5efd529ed85d None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:59.318640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4e6e0-3a72-4935-9f14-5efd529ed85d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:59.319325 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4e6e0-3a72-4935-9f14-5efd529ed85d None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:59.321004 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 873/1781] 10.222.0.25 () {54 vars in 912 bytes} [Mon May 9 18:29:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:29:59.328903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffa606-f8ad-4db5-b18e-9efc9a8e84cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:59.332573 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa606-f8ad-4db5-b18e-9efc9a8e84cd admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:59.334362 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa606-f8ad-4db5-b18e-9efc9a8e84cd admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:59.335285 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa606-f8ad-4db5-b18e-9efc9a8e84cd admin admin] PATH_INFO: `/v3/projects/9c91873b04b74fefb6aff1dc18a636be/users/d847a566cb9f405eac2026037979de11/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:59.339852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91ff7-228f-44cd-ac04-31131a8df2b6 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:59.340471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91ff7-228f-44cd-ac04-31131a8df2b6 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:59.340920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91ff7-228f-44cd-ac04-31131a8df2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:59.380479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-0ab91ff7-228f-44cd-ac04-31131a8df2b6 None None] MFA Rules not processed for user `6b2b6318776b422d9701351ba6b4ee27`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:59.382793 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-0ab91ff7-228f-44cd-ac04-31131a8df2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:59.387867 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffa606-f8ad-4db5-b18e-9efc9a8e84cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c91873b04b74fefb6aff1dc18a636be, user_id=d847a566cb9f405eac2026037979de11, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:59.390774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffa606-f8ad-4db5-b18e-9efc9a8e84cd admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:59.424191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 909/1782] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:29:59 2022] PUT /identity/v3/projects/9c91873b04b74fefb6aff1dc18a636be/users/d847a566cb9f405eac2026037979de11/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:29:59.434579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e289-dfe9-4b8d-986a-a031ae98736e None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:59.434866 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e289-dfe9-4b8d-986a-a031ae98736e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:59.435139 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e289-dfe9-4b8d-986a-a031ae98736e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:59.512870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-3506e289-dfe9-4b8d-986a-a031ae98736e None None] MFA Rules not processed for user `d847a566cb9f405eac2026037979de11`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:29:59.516384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3506e289-dfe9-4b8d-986a-a031ae98736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:59.550102 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 874/1783] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:29:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:29:59.620939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab91ff7-228f-44cd-ac04-31131a8df2b6 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:59.623056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-0ab91ff7-228f-44cd-ac04-31131a8df2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:29:59.643352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab91ff7-228f-44cd-ac04-31131a8df2b6 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:59.652219 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 910/1784] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:29:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:29:59.653441 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506e289-dfe9-4b8d-986a-a031ae98736e None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:59.773276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a4d0d-14e4-4949-b4cc-5a483f339c2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:59.774935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4d0d-14e4-4949-b4cc-5a483f339c2b service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:59.775465 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4d0d-14e4-4949-b4cc-5a483f339c2b service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:59.776667 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4d0d-14e4-4949-b4cc-5a483f339c2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:59.788034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a4d0d-14e4-4949-b4cc-5a483f339c2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:59.790765 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a4d0d-14e4-4949-b4cc-5a483f339c2b service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:59.793198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506e289-dfe9-4b8d-986a-a031ae98736e None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:59.798960 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5032d5c-2f72-4c3a-8ee8-cebfe90cb69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:59.800650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5032d5c-2f72-4c3a-8ee8-cebfe90cb69f admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:59.800892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5032d5c-2f72-4c3a-8ee8-cebfe90cb69f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:59.801215 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5032d5c-2f72-4c3a-8ee8-cebfe90cb69f admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:59.803272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5032d5c-2f72-4c3a-8ee8-cebfe90cb69f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:59.811652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 875/1785] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:29:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:29:59.819467 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5032d5c-2f72-4c3a-8ee8-cebfe90cb69f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:29:59.877630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 911/1786] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:29:59 2022] POST /identity/v3/projects => generated 381 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:29:59.880162 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a4d0d-14e4-4949-b4cc-5a483f339c2b service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:29:59.897135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a4d0d-14e4-4949-b4cc-5a483f339c2b service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:29:59.975117 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df0fdd-79e2-4b36-b585-228b79e2a118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:29:59.976101 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df0fdd-79e2-4b36-b585-228b79e2a118 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:29:59.976578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df0fdd-79e2-4b36-b585-228b79e2a118 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:29:59.977215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df0fdd-79e2-4b36-b585-228b79e2a118 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:29:59.978138 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df0fdd-79e2-4b36-b585-228b79e2a118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:29:59.980432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df0fdd-79e2-4b36-b585-228b79e2a118 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:00.069518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 876/1787] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:29:59 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:30:00.079903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5032d5c-2f72-4c3a-8ee8-cebfe90cb69f admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:00.112699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5032d5c-2f72-4c3a-8ee8-cebfe90cb69f admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:00.132257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdf69e-4da2-4aef-bebf-431dd507f6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:00.133529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdf69e-4da2-4aef-bebf-431dd507f6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:00.134021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdf69e-4da2-4aef-bebf-431dd507f6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:00.134529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdf69e-4da2-4aef-bebf-431dd507f6e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:00.135573 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdf69e-4da2-4aef-bebf-431dd507f6e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:00.138012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdf69e-4da2-4aef-bebf-431dd507f6e1 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:00.161339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 912/1788] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:00.165863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df0fdd-79e2-4b36-b585-228b79e2a118 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:00.185156 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df0fdd-79e2-4b36-b585-228b79e2a118 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:00.199401 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc799cd-41c8-4411-9f68-cc25c991a25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:00.206856 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc799cd-41c8-4411-9f68-cc25c991a25e admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:00.207413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc799cd-41c8-4411-9f68-cc25c991a25e admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:00.211213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc799cd-41c8-4411-9f68-cc25c991a25e admin admin] PATH_INFO: `/v3/projects/3f33e0f1501e4949b413c59dbbeaf24a/users/78c926796cc5409a9244540c732b6d67/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:00.251351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc799cd-41c8-4411-9f68-cc25c991a25e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f33e0f1501e4949b413c59dbbeaf24a, user_id=78c926796cc5409a9244540c732b6d67, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:00.254027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc799cd-41c8-4411-9f68-cc25c991a25e admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:00.292392 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 877/1789] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:00 2022] PUT /identity/v3/projects/3f33e0f1501e4949b413c59dbbeaf24a/users/78c926796cc5409a9244540c732b6d67/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:00.298249 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdf69e-4da2-4aef-bebf-431dd507f6e1 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:00.309964 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdf69e-4da2-4aef-bebf-431dd507f6e1 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:00.492671 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ac823-080a-43a9-9128-d130a1a40f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:00.493362 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ac823-080a-43a9-9128-d130a1a40f5a admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:00.493786 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ac823-080a-43a9-9128-d130a1a40f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:00.494060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ac823-080a-43a9-9128-d130a1a40f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:00.494783 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ac823-080a-43a9-9128-d130a1a40f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:00.496423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ac823-080a-43a9-9128-d130a1a40f5a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:00.523478 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 913/1790] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:00.528579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fab821-68df-4154-a1cd-46985189bb0c None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:00.529496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fab821-68df-4154-a1cd-46985189bb0c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:00.529925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ac823-080a-43a9-9128-d130a1a40f5a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:00.530220 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fab821-68df-4154-a1cd-46985189bb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:00.543935 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ac823-080a-43a9-9128-d130a1a40f5a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:00.553113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142dfe2-92ab-44b3-81b9-d3974c55119b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:00.556144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142dfe2-92ab-44b3-81b9-d3974c55119b admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:00.557190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142dfe2-92ab-44b3-81b9-d3974c55119b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:00.557629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142dfe2-92ab-44b3-81b9-d3974c55119b admin admin] PATH_INFO: `/v3/projects/3f33e0f1501e4949b413c59dbbeaf24a/users/78c926796cc5409a9244540c732b6d67/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:00.562001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142dfe2-92ab-44b3-81b9-d3974c55119b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f33e0f1501e4949b413c59dbbeaf24a, user_id=78c926796cc5409a9244540c732b6d67, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:00.563142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-21fab821-68df-4154-a1cd-46985189bb0c None None] MFA Rules not processed for user `90fdc96d6e6649a498bb3a8659fcc030`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:00.564676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-21fab821-68df-4154-a1cd-46985189bb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:00.566316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142dfe2-92ab-44b3-81b9-d3974c55119b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:00.580809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 914/1791] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:00 2022] PUT /identity/v3/projects/3f33e0f1501e4949b413c59dbbeaf24a/users/78c926796cc5409a9244540c732b6d67/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:00.593911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be3a7-158c-4885-9456-8f23d76b7587 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:00.597365 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be3a7-158c-4885-9456-8f23d76b7587 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:00.602369 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be3a7-158c-4885-9456-8f23d76b7587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:00.712510 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-ab0be3a7-158c-4885-9456-8f23d76b7587 None None] MFA Rules not processed for user `78c926796cc5409a9244540c732b6d67`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:00.714151 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ab0be3a7-158c-4885-9456-8f23d76b7587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:00.780353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 878/1792] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:00.780353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fab821-68df-4154-a1cd-46985189bb0c None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:00.780353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-21fab821-68df-4154-a1cd-46985189bb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:00.900626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 915/1793] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:00.923423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be3a7-158c-4885-9456-8f23d76b7587 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:00.925520 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ab0be3a7-158c-4885-9456-8f23d76b7587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:00.976212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fab821-68df-4154-a1cd-46985189bb0c None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:01.110691 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48106199-c1e3-4bed-b03e-f481b478e969 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:01.111339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be3a7-158c-4885-9456-8f23d76b7587 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:01.111738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48106199-c1e3-4bed-b03e-f481b478e969 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:01.112024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48106199-c1e3-4bed-b03e-f481b478e969 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:01.113700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48106199-c1e3-4bed-b03e-f481b478e969 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:01.121831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-48106199-c1e3-4bed-b03e-f481b478e969 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:01.123739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb509b0d-652a-463a-8bc7-b67d410f04e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:01.124576 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb509b0d-652a-463a-8bc7-b67d410f04e2 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:01.124867 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb509b0d-652a-463a-8bc7-b67d410f04e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:01.125150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb509b0d-652a-463a-8bc7-b67d410f04e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:01.148086 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48106199-c1e3-4bed-b03e-f481b478e969 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:01.149837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48106199-c1e3-4bed-b03e-f481b478e969 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:01.155900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb509b0d-652a-463a-8bc7-b67d410f04e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:01.157345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb509b0d-652a-463a-8bc7-b67d410f04e2 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:01.183891 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 879/1794] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:01.186951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48106199-c1e3-4bed-b03e-f481b478e969 service nova] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:01.192226 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-48106199-c1e3-4bed-b03e-f481b478e969 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:01.228628 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 916/1795] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:30:01.327683 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf498b6-fa6d-4a54-919f-6243b5d3362b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:01.328321 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf498b6-fa6d-4a54-919f-6243b5d3362b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:01.328321 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf498b6-fa6d-4a54-919f-6243b5d3362b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:01.373973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-bcf498b6-fa6d-4a54-919f-6243b5d3362b None None] MFA Rules not processed for user `acae3ad19a7e45b9877520dff8f5b995`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:01.376251 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-bcf498b6-fa6d-4a54-919f-6243b5d3362b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:01.424301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48106199-c1e3-4bed-b03e-f481b478e969 service nova] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:01.434769 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:01.444627 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:01.444930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:01.445238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] PATH_INFO: `/v3/projects/3413dab8ad594356ad8cde1976e33954` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:01.449446 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3413dab8ad594356ad8cde1976e33954)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:01.466920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:01.484993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 917/1796] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:01.616104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941aa40-91cb-417b-b467-45b81bacb535 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:01.616515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941aa40-91cb-417b-b467-45b81bacb535 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:01.617062 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941aa40-91cb-417b-b467-45b81bacb535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:01.671818 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3413dab8ad594356ad8cde1976e33954', 'request_id': 'req-8f818013-4758-4af8-b7d9-eef130619278'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:01.690703 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3413dab8ad594356ad8cde1976e33954', 'request_id': 'req-8f818013-4758-4af8-b7d9-eef130619278'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:01.709681 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4941aa40-91cb-417b-b467-45b81bacb535 None None] MFA Rules not processed for user `c1d626990b114bf3bd19bd6386831bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:01.711453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3413dab8ad594356ad8cde1976e33954', 'request_id': 'req-8f818013-4758-4af8-b7d9-eef130619278'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:01.722249 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4941aa40-91cb-417b-b467-45b81bacb535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:01.747938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] The token cache is being invalidate because project 3413dab8ad594356ad8cde1976e33954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:01.748509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:01.750486 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:01.756273 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 880/1797] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:01 2022] DELETE /identity/v3/projects/3413dab8ad594356ad8cde1976e33954 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:01.760481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:01.764227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:02.081270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 918/1798] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:02.084262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941aa40-91cb-417b-b467-45b81bacb535 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:02.087063 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4941aa40-91cb-417b-b467-45b81bacb535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:02.102921 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f818013-4758-4af8-b7d9-eef130619278 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:02.118050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590fdbf-6e12-4086-b604-b823f588d91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:02.119925 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590fdbf-6e12-4086-b604-b823f588d91a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:02.120549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590fdbf-6e12-4086-b604-b823f588d91a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:02.129025 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590fdbf-6e12-4086-b604-b823f588d91a admin admin] PATH_INFO: `/v3/users/9d868a7682d2464c8b163ae311dc13e9` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:02.131211 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941aa40-91cb-417b-b467-45b81bacb535 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:02.132107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590fdbf-6e12-4086-b604-b823f588d91a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d868a7682d2464c8b163ae311dc13e9)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:02.135639 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590fdbf-6e12-4086-b604-b823f588d91a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:02.153470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e3c8e-21af-4b26-a2ea-71631399b897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:02.155264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e3c8e-21af-4b26-a2ea-71631399b897 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:02.161299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e3c8e-21af-4b26-a2ea-71631399b897 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:02.162552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e3c8e-21af-4b26-a2ea-71631399b897 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:02.165967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e3c8e-21af-4b26-a2ea-71631399b897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:02.168078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e3c8e-21af-4b26-a2ea-71631399b897 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:02.281038 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 919/1799] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:02 2022] POST /identity/v3/projects => generated 371 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:02.283462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e3c8e-21af-4b26-a2ea-71631399b897 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:02.288090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-526e3c8e-21af-4b26-a2ea-71631399b897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:02.305339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d590fdbf-6e12-4086-b604-b823f588d91a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d868a7682d2464c8b163ae311dc13e9'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:02.320824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d590fdbf-6e12-4086-b604-b823f588d91a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d868a7682d2464c8b163ae311dc13e9'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:02.322669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d590fdbf-6e12-4086-b604-b823f588d91a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d868a7682d2464c8b163ae311dc13e9'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:02.339434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d590fdbf-6e12-4086-b604-b823f588d91a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d868a7682d2464c8b163ae311dc13e9'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:02.340250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d590fdbf-6e12-4086-b604-b823f588d91a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d868a7682d2464c8b163ae311dc13e9'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:02.399628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 881/1800] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:01 2022] DELETE /identity/v3/users/9d868a7682d2464c8b163ae311dc13e9 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:02.402613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590fdbf-6e12-4086-b604-b823f588d91a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:02.415151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-d590fdbf-6e12-4086-b604-b823f588d91a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:02.519084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e3c8e-21af-4b26-a2ea-71631399b897 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:02.647772 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590fdbf-6e12-4086-b604-b823f588d91a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:02.658912 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292cce5d-a37b-4635-b9d6-5d85ef47e83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:02.660772 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cce5d-a37b-4635-b9d6-5d85ef47e83a admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:02.661162 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cce5d-a37b-4635-b9d6-5d85ef47e83a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:02.662366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cce5d-a37b-4635-b9d6-5d85ef47e83a admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:02.665037 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292cce5d-a37b-4635-b9d6-5d85ef47e83a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:02.672333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292cce5d-a37b-4635-b9d6-5d85ef47e83a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:02.691496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba95adc-88fd-4215-9f44-2377feec1282 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c926796cc5409a9244540c732b6d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f33e0f1501e4949b413c59dbbeaf24a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:02.692773 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba95adc-88fd-4215-9f44-2377feec1282 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:02.693596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba95adc-88fd-4215-9f44-2377feec1282 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:02.694141 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba95adc-88fd-4215-9f44-2377feec1282 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:02.695516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba95adc-88fd-4215-9f44-2377feec1282 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:30:02.697232 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba95adc-88fd-4215-9f44-2377feec1282 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:02.712855 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba95adc-88fd-4215-9f44-2377feec1282 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:02.729153 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 882/1801] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:02 2022] POST /identity/v3/users => generated 292 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:02.733240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292cce5d-a37b-4635-b9d6-5d85ef47e83a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:02.738719 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 920/1802] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:30:02.740317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba95adc-88fd-4215-9f44-2377feec1282 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:02.742082 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-0ba95adc-88fd-4215-9f44-2377feec1282 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:02.759438 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292cce5d-a37b-4635-b9d6-5d85ef47e83a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:02.768375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba95adc-88fd-4215-9f44-2377feec1282 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:02.774208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b2041-72f0-4058-9dbc-b62921974763 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c926796cc5409a9244540c732b6d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f33e0f1501e4949b413c59dbbeaf24a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:02.775194 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b2041-72f0-4058-9dbc-b62921974763 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:02.775599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b2041-72f0-4058-9dbc-b62921974763 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:02.776748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21109f2c-a0cf-4975-b53c-36f9a7bdb67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:02.776933 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b2041-72f0-4058-9dbc-b62921974763 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:02.777761 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b2041-72f0-4058-9dbc-b62921974763 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:02.777878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21109f2c-a0cf-4975-b53c-36f9a7bdb67a admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:02.778282 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21109f2c-a0cf-4975-b53c-36f9a7bdb67a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:02.778717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21109f2c-a0cf-4975-b53c-36f9a7bdb67a admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:02.779695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21109f2c-a0cf-4975-b53c-36f9a7bdb67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:02.781619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21109f2c-a0cf-4975-b53c-36f9a7bdb67a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:02.781752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b2041-72f0-4058-9dbc-b62921974763 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:02.818915 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 883/1803] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:02.822877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21109f2c-a0cf-4975-b53c-36f9a7bdb67a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:02.838187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 921/1804] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:02 2022] POST /identity/v3/projects => generated 354 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:02.841043 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21109f2c-a0cf-4975-b53c-36f9a7bdb67a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:02.857296 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b2041-72f0-4058-9dbc-b62921974763 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:02.990073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1cc97-7528-4452-886b-8d783598b116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:02.991330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1cc97-7528-4452-886b-8d783598b116 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:02.991760 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1cc97-7528-4452-886b-8d783598b116 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:02.992194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1cc97-7528-4452-886b-8d783598b116 admin admin] PATH_INFO: `/v3/projects/3a7871c956d24192801f52ee5e3692e8/users/7ee8d752027e4d14b7344ec32b95fb4e/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:03.023550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1cc97-7528-4452-886b-8d783598b116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a7871c956d24192801f52ee5e3692e8, user_id=7ee8d752027e4d14b7344ec32b95fb4e, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:03.030387 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1cc97-7528-4452-886b-8d783598b116 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:03.041024 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b2041-72f0-4058-9dbc-b62921974763 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:03.056282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a53a6-f7bb-4a3d-9d23-408d60edf1ca tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c926796cc5409a9244540c732b6d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f33e0f1501e4949b413c59dbbeaf24a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:03.057822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a53a6-f7bb-4a3d-9d23-408d60edf1ca tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:03.058179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a53a6-f7bb-4a3d-9d23-408d60edf1ca tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:03.058531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a53a6-f7bb-4a3d-9d23-408d60edf1ca tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:03.059603 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 884/1805] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:02 2022] PUT /identity/v3/projects/3a7871c956d24192801f52ee5e3692e8/users/7ee8d752027e4d14b7344ec32b95fb4e/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:03.059711 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a53a6-f7bb-4a3d-9d23-408d60edf1ca tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:30:03.060063 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a53a6-f7bb-4a3d-9d23-408d60edf1ca tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:03.060990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1cc97-7528-4452-886b-8d783598b116 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:03.061772 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a53a6-f7bb-4a3d-9d23-408d60edf1ca tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:03.072781 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c8c1cc97-7528-4452-886b-8d783598b116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:03.093271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 922/1806] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:30:03.093271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a53a6-f7bb-4a3d-9d23-408d60edf1ca tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:03.113869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a53a6-f7bb-4a3d-9d23-408d60edf1ca tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:03.196675 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1cc97-7528-4452-886b-8d783598b116 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:03.202442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:03.203740 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:03.204398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:03.205472 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] PATH_INFO: `/v3/projects/0755853c51354cf7aa942f6bdde3c948` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:03.206745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0755853c51354cf7aa942f6bdde3c948)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:03.208986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:03.234034 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0aa1e-c705-4cde-bf0c-0b51b036e80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:03.234805 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0aa1e-c705-4cde-bf0c-0b51b036e80f admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:03.235087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0aa1e-c705-4cde-bf0c-0b51b036e80f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:03.235410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0aa1e-c705-4cde-bf0c-0b51b036e80f admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:03.236145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0aa1e-c705-4cde-bf0c-0b51b036e80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:03.238155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0aa1e-c705-4cde-bf0c-0b51b036e80f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:03.253426 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 923/1807] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:03.254479 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0aa1e-c705-4cde-bf0c-0b51b036e80f admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:03.287842 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0755853c51354cf7aa942f6bdde3c948', 'request_id': 'req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:03.296718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0755853c51354cf7aa942f6bdde3c948', 'request_id': 'req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:03.309692 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0755853c51354cf7aa942f6bdde3c948', 'request_id': 'req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:03.341527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0aa1e-c705-4cde-bf0c-0b51b036e80f admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:03.345170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] The token cache is being invalidate because project 0755853c51354cf7aa942f6bdde3c948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:03.345471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:03.348904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:03.357534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 885/1808] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:03 2022] DELETE /identity/v3/projects/0755853c51354cf7aa942f6bdde3c948 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:03.358270 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:03.364766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:03.446875 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d19dbe-e189-4f3e-b240-88f741f01476 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c926796cc5409a9244540c732b6d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f33e0f1501e4949b413c59dbbeaf24a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:03.448340 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d19dbe-e189-4f3e-b240-88f741f01476 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:03.448340 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d19dbe-e189-4f3e-b240-88f741f01476 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:03.448758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d19dbe-e189-4f3e-b240-88f741f01476 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] PATH_INFO: `/v3/projects/93f9425d001042b3914d3a93f4b428cf` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:03.452563 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d19dbe-e189-4f3e-b240-88f741f01476 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] RBAC: Authorizing `identity:delete_project(project_id=93f9425d001042b3914d3a93f4b428cf)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:03.454645 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d19dbe-e189-4f3e-b240-88f741f01476 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:03.510783 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd0ae4-1ba4-48b0-a318-4d55239fb21c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:03.521073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854d673-0cb8-4946-9369-b5f50d642b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:03.524387 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854d673-0cb8-4946-9369-b5f50d642b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:03.524710 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854d673-0cb8-4946-9369-b5f50d642b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:03.524952 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854d673-0cb8-4946-9369-b5f50d642b9b admin admin] PATH_INFO: `/v3/projects/3a7871c956d24192801f52ee5e3692e8/users/7ee8d752027e4d14b7344ec32b95fb4e/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:03.529356 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854d673-0cb8-4946-9369-b5f50d642b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a7871c956d24192801f52ee5e3692e8, user_id=7ee8d752027e4d14b7344ec32b95fb4e, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:03.532473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854d673-0cb8-4946-9369-b5f50d642b9b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:03.543840 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 886/1809] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:03 2022] PUT /identity/v3/projects/3a7871c956d24192801f52ee5e3692e8/users/7ee8d752027e4d14b7344ec32b95fb4e/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:03.545194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac358aea-e0ac-4ce7-a245-90daed6dcfc7 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:03.545487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac358aea-e0ac-4ce7-a245-90daed6dcfc7 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:03.545769 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac358aea-e0ac-4ce7-a245-90daed6dcfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:03.549664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d7d19dbe-e189-4f3e-b240-88f741f01476 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93f9425d001042b3914d3a93f4b428cf', 'request_id': 'req-d7d19dbe-e189-4f3e-b240-88f741f01476'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:03.556763 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d7d19dbe-e189-4f3e-b240-88f741f01476 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93f9425d001042b3914d3a93f4b428cf', 'request_id': 'req-d7d19dbe-e189-4f3e-b240-88f741f01476'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:03.569902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d7d19dbe-e189-4f3e-b240-88f741f01476 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93f9425d001042b3914d3a93f4b428cf', 'request_id': 'req-d7d19dbe-e189-4f3e-b240-88f741f01476'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:03.586296 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d7d19dbe-e189-4f3e-b240-88f741f01476 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] The token cache is being invalidate because project 93f9425d001042b3914d3a93f4b428cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:03.586592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d7d19dbe-e189-4f3e-b240-88f741f01476 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:03.587391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d7d19dbe-e189-4f3e-b240-88f741f01476 tempest-VolumeQuotaClassesTest-1640439958 tempest-VolumeQuotaClassesTest-1640439958-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:03.587736 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ac358aea-e0ac-4ce7-a245-90daed6dcfc7 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:03.590669 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 924/1810] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:03 2022] DELETE /identity/v3/projects/93f9425d001042b3914d3a93f4b428cf => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:03.590826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ac358aea-e0ac-4ce7-a245-90daed6dcfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:03.700265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9186aa-2210-43fe-beb3-5e8eff870f14 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:03.701466 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9186aa-2210-43fe-beb3-5e8eff870f14 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:03.701944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9186aa-2210-43fe-beb3-5e8eff870f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:03.705626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 887/1811] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:03.730454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-0c9186aa-2210-43fe-beb3-5e8eff870f14 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:03.732269 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-0c9186aa-2210-43fe-beb3-5e8eff870f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:03.747718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac358aea-e0ac-4ce7-a245-90daed6dcfc7 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:03.748546 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 925/1812] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:03.781809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac358aea-e0ac-4ce7-a245-90daed6dcfc7 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:03.790814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0453a-944f-4d3a-89c5-ed5e35e9a128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:03.792633 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0453a-944f-4d3a-89c5-ed5e35e9a128 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:03.793132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0453a-944f-4d3a-89c5-ed5e35e9a128 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:03.793640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0453a-944f-4d3a-89c5-ed5e35e9a128 admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:03.794649 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0453a-944f-4d3a-89c5-ed5e35e9a128 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:30:03.798274 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0453a-944f-4d3a-89c5-ed5e35e9a128 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:03.800433 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0453a-944f-4d3a-89c5-ed5e35e9a128 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:03.823254 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 888/1813] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:30:03.827238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9186aa-2210-43fe-beb3-5e8eff870f14 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:03.847211 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9186aa-2210-43fe-beb3-5e8eff870f14 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:03.854400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfd517-0e39-4215-a1dd-d49e3152cdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:03.855425 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfd517-0e39-4215-a1dd-d49e3152cdae admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:03.855825 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfd517-0e39-4215-a1dd-d49e3152cdae admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:03.856191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfd517-0e39-4215-a1dd-d49e3152cdae admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:03.857054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfd517-0e39-4215-a1dd-d49e3152cdae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:03.860841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfd517-0e39-4215-a1dd-d49e3152cdae admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:03.925088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0453a-944f-4d3a-89c5-ed5e35e9a128 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:03.926878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-d4d0453a-944f-4d3a-89c5-ed5e35e9a128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:03.941490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0453a-944f-4d3a-89c5-ed5e35e9a128 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:03.946821 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e04a6-aff1-45bc-8ab9-a99ef54eaf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:03.947741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e04a6-aff1-45bc-8ab9-a99ef54eaf46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:03.948081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e04a6-aff1-45bc-8ab9-a99ef54eaf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:03.948290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e04a6-aff1-45bc-8ab9-a99ef54eaf46 admin admin] PATH_INFO: `/v3/users/d847a566cb9f405eac2026037979de11` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:03.950673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e04a6-aff1-45bc-8ab9-a99ef54eaf46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d847a566cb9f405eac2026037979de11)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:03.952697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e04a6-aff1-45bc-8ab9-a99ef54eaf46 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:03.960310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 926/1814] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:03 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:03.965579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfd517-0e39-4215-a1dd-d49e3152cdae admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:04.064902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-029e04a6-aff1-45bc-8ab9-a99ef54eaf46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd847a566cb9f405eac2026037979de11'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:04.083666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-029e04a6-aff1-45bc-8ab9-a99ef54eaf46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd847a566cb9f405eac2026037979de11'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:04.084496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-029e04a6-aff1-45bc-8ab9-a99ef54eaf46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd847a566cb9f405eac2026037979de11'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:04.097272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-029e04a6-aff1-45bc-8ab9-a99ef54eaf46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd847a566cb9f405eac2026037979de11'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:04.098978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-029e04a6-aff1-45bc-8ab9-a99ef54eaf46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd847a566cb9f405eac2026037979de11'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:04.162108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 889/1815] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:03 2022] DELETE /identity/v3/users/d847a566cb9f405eac2026037979de11 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:04.168780 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d47e80-f6bd-48c6-b03a-6705995f59fa None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:04.169660 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d47e80-f6bd-48c6-b03a-6705995f59fa None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:04.169960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d47e80-f6bd-48c6-b03a-6705995f59fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:04.195618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfd517-0e39-4215-a1dd-d49e3152cdae admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:04.222860 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-b8d47e80-f6bd-48c6-b03a-6705995f59fa None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:04.225138 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b8d47e80-f6bd-48c6-b03a-6705995f59fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:04.311417 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 890/1816] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:04.338755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d47e80-f6bd-48c6-b03a-6705995f59fa None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:04.341240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b8d47e80-f6bd-48c6-b03a-6705995f59fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:04.378820 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac4e51-3191-4833-9dc6-90325af33df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:04.379559 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac4e51-3191-4833-9dc6-90325af33df4 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:04.379826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac4e51-3191-4833-9dc6-90325af33df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:04.380106 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac4e51-3191-4833-9dc6-90325af33df4 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:04.380803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac4e51-3191-4833-9dc6-90325af33df4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:04.382891 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac4e51-3191-4833-9dc6-90325af33df4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:04.452462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 927/1817] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:03 2022] POST /identity/v3/users => generated 293 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:30:04.454176 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac4e51-3191-4833-9dc6-90325af33df4 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:04.461360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-feac4e51-3191-4833-9dc6-90325af33df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:04.475973 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d47e80-f6bd-48c6-b03a-6705995f59fa None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:04.486445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6d161-6f80-47fb-8dfe-b23a662dcdc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:04.488742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6d161-6f80-47fb-8dfe-b23a662dcdc4 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:04.489037 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6d161-6f80-47fb-8dfe-b23a662dcdc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:04.489755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6d161-6f80-47fb-8dfe-b23a662dcdc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:04.496241 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac4e51-3191-4833-9dc6-90325af33df4 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:04.523244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e282ee-4ea7-4d9d-9f14-7d3b129c80ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:04.523949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e282ee-4ea7-4d9d-9f14-7d3b129c80ae admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:04.524234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e282ee-4ea7-4d9d-9f14-7d3b129c80ae admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:04.524510 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e282ee-4ea7-4d9d-9f14-7d3b129c80ae admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:04.525253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e282ee-4ea7-4d9d-9f14-7d3b129c80ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:04.526844 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e282ee-4ea7-4d9d-9f14-7d3b129c80ae admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:04.548838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6d161-6f80-47fb-8dfe-b23a662dcdc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:04.552943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6d161-6f80-47fb-8dfe-b23a662dcdc4 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:04.567879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 928/1818] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:04.568836 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e282ee-4ea7-4d9d-9f14-7d3b129c80ae admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:04.584162 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 891/1819] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:30:04.598310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e282ee-4ea7-4d9d-9f14-7d3b129c80ae admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:04.614499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d1f0f-fd1e-41a8-89dc-497ec595f214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:04.615220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d1f0f-fd1e-41a8-89dc-497ec595f214 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:04.615499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d1f0f-fd1e-41a8-89dc-497ec595f214 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:04.615778 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d1f0f-fd1e-41a8-89dc-497ec595f214 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:04.616528 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d1f0f-fd1e-41a8-89dc-497ec595f214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:04.618090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d1f0f-fd1e-41a8-89dc-497ec595f214 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:04.664868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6d161-6f80-47fb-8dfe-b23a662dcdc4 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:04.671599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 929/1820] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:04.683907 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6d161-6f80-47fb-8dfe-b23a662dcdc4 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:04.707572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aee05c-4be6-47e8-81b7-a5521811037a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:04.709419 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aee05c-4be6-47e8-81b7-a5521811037a admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:04.711458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aee05c-4be6-47e8-81b7-a5521811037a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:04.713176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aee05c-4be6-47e8-81b7-a5521811037a admin admin] PATH_INFO: `/v3/projects/93ecde25b0754b23a8a40ee9ab9219c3/users/01c5c4a7b0524952a904cf229073a953/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:04.787978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aee05c-4be6-47e8-81b7-a5521811037a admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ecde25b0754b23a8a40ee9ab9219c3, user_id=01c5c4a7b0524952a904cf229073a953, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:04.798632 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aee05c-4be6-47e8-81b7-a5521811037a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:04.846061 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 892/1821] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:04 2022] PUT /identity/v3/projects/93ecde25b0754b23a8a40ee9ab9219c3/users/01c5c4a7b0524952a904cf229073a953/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:04.860918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef799d4-532e-4d3a-91c2-85fd1d43b641 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:04.861637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef799d4-532e-4d3a-91c2-85fd1d43b641 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:04.862385 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef799d4-532e-4d3a-91c2-85fd1d43b641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:04.919203 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2ef799d4-532e-4d3a-91c2-85fd1d43b641 None None] MFA Rules not processed for user `01c5c4a7b0524952a904cf229073a953`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:04.922794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2ef799d4-532e-4d3a-91c2-85fd1d43b641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:05.091951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aee05c-4be6-47e8-81b7-a5521811037a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:05.092705 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 930/1822] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:05.215132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aee05c-4be6-47e8-81b7-a5521811037a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:05.241269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317a3a8-efd4-43c4-aa48-a20d00a50e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:05.241964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317a3a8-efd4-43c4-aa48-a20d00a50e20 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:05.242251 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317a3a8-efd4-43c4-aa48-a20d00a50e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:05.242603 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317a3a8-efd4-43c4-aa48-a20d00a50e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:05.243400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317a3a8-efd4-43c4-aa48-a20d00a50e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:05.245719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317a3a8-efd4-43c4-aa48-a20d00a50e20 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:05.300555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef799d4-532e-4d3a-91c2-85fd1d43b641 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:05.309929 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 893/1823] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:05 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:05.316559 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef799d4-532e-4d3a-91c2-85fd1d43b641 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:05.513766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a1b2c-9326-4336-b339-57458613620b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:05.514523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a1b2c-9326-4336-b339-57458613620b admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:05.514796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a1b2c-9326-4336-b339-57458613620b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:05.515523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a1b2c-9326-4336-b339-57458613620b admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:05.517032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a1b2c-9326-4336-b339-57458613620b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:05.522363 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a1b2c-9326-4336-b339-57458613620b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:05.560916 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 931/1824] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:05 2022] POST /identity/v3/users => generated 294 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:05.568780 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317a3a8-efd4-43c4-aa48-a20d00a50e20 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:05.609425 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317a3a8-efd4-43c4-aa48-a20d00a50e20 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:05.617863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67c3c1-ee16-4fab-901d-5640c5a8f52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:05.618561 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67c3c1-ee16-4fab-901d-5640c5a8f52b admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:05.618887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67c3c1-ee16-4fab-901d-5640c5a8f52b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:05.619165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67c3c1-ee16-4fab-901d-5640c5a8f52b admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:05.620073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67c3c1-ee16-4fab-901d-5640c5a8f52b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:05.623028 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67c3c1-ee16-4fab-901d-5640c5a8f52b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:05.642061 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 894/1825] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:05.646814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a1b2c-9326-4336-b339-57458613620b admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:05.660663 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a1b2c-9326-4336-b339-57458613620b admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:05.666570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83344de4-189e-4735-9f2f-d64799d243f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:05.667478 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344de4-189e-4735-9f2f-d64799d243f6 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:05.667851 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344de4-189e-4735-9f2f-d64799d243f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:05.668309 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344de4-189e-4735-9f2f-d64799d243f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:05.669454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83344de4-189e-4735-9f2f-d64799d243f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:05.671664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83344de4-189e-4735-9f2f-d64799d243f6 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:05.688513 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 932/1826] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:05.688705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67c3c1-ee16-4fab-901d-5640c5a8f52b admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:05.720806 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67c3c1-ee16-4fab-901d-5640c5a8f52b admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:05.736052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b2e74-4229-46da-8b8c-9589cbef85ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:05.736983 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b2e74-4229-46da-8b8c-9589cbef85ef admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:05.737742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b2e74-4229-46da-8b8c-9589cbef85ef admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:05.752995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b2e74-4229-46da-8b8c-9589cbef85ef admin admin] PATH_INFO: `/v3/projects/3e3c8e07e00841d385759c2012714bd8/users/aa687bdff8b14523a839024ecd8eb5a4/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:05.830974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b2e74-4229-46da-8b8c-9589cbef85ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e3c8e07e00841d385759c2012714bd8, user_id=aa687bdff8b14523a839024ecd8eb5a4, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:05.856617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b2e74-4229-46da-8b8c-9589cbef85ef admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:05.917965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 895/1827] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:05 2022] PUT /identity/v3/projects/3e3c8e07e00841d385759c2012714bd8/users/aa687bdff8b14523a839024ecd8eb5a4/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:05.931696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d7a55-cbe6-4c40-9ba6-54c3b6ffac6d None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:05.932268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d7a55-cbe6-4c40-9ba6-54c3b6ffac6d None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:05.932719 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d7a55-cbe6-4c40-9ba6-54c3b6ffac6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:05.999972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-267d7a55-cbe6-4c40-9ba6-54c3b6ffac6d None None] MFA Rules not processed for user `aa687bdff8b14523a839024ecd8eb5a4`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:06.002973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-267d7a55-cbe6-4c40-9ba6-54c3b6ffac6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:06.016733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b2e74-4229-46da-8b8c-9589cbef85ef admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:06.024969 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-607b2e74-4229-46da-8b8c-9589cbef85ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:06.136674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 933/1828] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:06.137846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d7a55-cbe6-4c40-9ba6-54c3b6ffac6d None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:06.230110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b2e74-4229-46da-8b8c-9589cbef85ef admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:06.249061 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5717c69-6c29-473a-ab10-a3236a68da4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:06.251498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5717c69-6c29-473a-ab10-a3236a68da4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:06.251498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5717c69-6c29-473a-ab10-a3236a68da4f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:06.251498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5717c69-6c29-473a-ab10-a3236a68da4f admin admin] PATH_INFO: `/v3/projects/9c91873b04b74fefb6aff1dc18a636be` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:06.258338 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5717c69-6c29-473a-ab10-a3236a68da4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c91873b04b74fefb6aff1dc18a636be)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:06.260589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5717c69-6c29-473a-ab10-a3236a68da4f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:06.298829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d7a55-cbe6-4c40-9ba6-54c3b6ffac6d None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:06.306423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c6697-baef-490d-b52a-6c2e9ce8f44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:06.307419 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c6697-baef-490d-b52a-6c2e9ce8f44f admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:06.307900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c6697-baef-490d-b52a-6c2e9ce8f44f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:06.308319 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c6697-baef-490d-b52a-6c2e9ce8f44f admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:06.309153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c6697-baef-490d-b52a-6c2e9ce8f44f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:06.312670 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c6697-baef-490d-b52a-6c2e9ce8f44f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:06.373164 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 934/1829] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:06 2022] POST /identity/v3/projects => generated 385 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:06.374891 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c6697-baef-490d-b52a-6c2e9ce8f44f admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:06.446803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b5717c69-6c29-473a-ab10-a3236a68da4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c91873b04b74fefb6aff1dc18a636be', 'request_id': 'req-b5717c69-6c29-473a-ab10-a3236a68da4f'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:06.459441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b5717c69-6c29-473a-ab10-a3236a68da4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c91873b04b74fefb6aff1dc18a636be', 'request_id': 'req-b5717c69-6c29-473a-ab10-a3236a68da4f'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:06.504624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b5717c69-6c29-473a-ab10-a3236a68da4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c91873b04b74fefb6aff1dc18a636be', 'request_id': 'req-b5717c69-6c29-473a-ab10-a3236a68da4f'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:06.536990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b5717c69-6c29-473a-ab10-a3236a68da4f admin admin] The token cache is being invalidate because project 9c91873b04b74fefb6aff1dc18a636be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:06.537432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b5717c69-6c29-473a-ab10-a3236a68da4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:06.542923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b5717c69-6c29-473a-ab10-a3236a68da4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:06.550607 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 896/1830] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:06 2022] DELETE /identity/v3/projects/9c91873b04b74fefb6aff1dc18a636be => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:06.554392 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c46c0-c511-49b9-8bbe-4eac6dba9ae5 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:06.558773 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c46c0-c511-49b9-8bbe-4eac6dba9ae5 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:06.559312 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c46c0-c511-49b9-8bbe-4eac6dba9ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:06.569438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c6697-baef-490d-b52a-6c2e9ce8f44f admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:06.669150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-9b4c46c0-c511-49b9-8bbe-4eac6dba9ae5 None None] MFA Rules not processed for user `7ee8d752027e4d14b7344ec32b95fb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:06.678579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9b4c46c0-c511-49b9-8bbe-4eac6dba9ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:06.702897 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5707432-1ad0-4c36-a6f0-7878f85745e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:06.703628 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5707432-1ad0-4c36-a6f0-7878f85745e0 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:06.703790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5707432-1ad0-4c36-a6f0-7878f85745e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:06.704068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5707432-1ad0-4c36-a6f0-7878f85745e0 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:06.704759 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5707432-1ad0-4c36-a6f0-7878f85745e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:06.714341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5707432-1ad0-4c36-a6f0-7878f85745e0 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:06.755903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 935/1831] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:06 2022] POST /identity/v3/users => generated 299 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:06.757500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5707432-1ad0-4c36-a6f0-7878f85745e0 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:06.759395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f5707432-1ad0-4c36-a6f0-7878f85745e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:06.777496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5707432-1ad0-4c36-a6f0-7878f85745e0 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:06.790321 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd203af-2d11-4201-8ef0-1c462ae01cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:06.791470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd203af-2d11-4201-8ef0-1c462ae01cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:06.791825 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd203af-2d11-4201-8ef0-1c462ae01cff admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:06.792400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd203af-2d11-4201-8ef0-1c462ae01cff admin admin] PATH_INFO: `/v3/users/78c926796cc5409a9244540c732b6d67` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:06.796036 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd203af-2d11-4201-8ef0-1c462ae01cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=78c926796cc5409a9244540c732b6d67)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:06.800005 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd203af-2d11-4201-8ef0-1c462ae01cff admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:06.878007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 897/1832] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:06 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:06.883678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c46c0-c511-49b9-8bbe-4eac6dba9ae5 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:06.885290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9b4c46c0-c511-49b9-8bbe-4eac6dba9ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:06.906970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c46c0-c511-49b9-8bbe-4eac6dba9ae5 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:06.909953 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-6cd203af-2d11-4201-8ef0-1c462ae01cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78c926796cc5409a9244540c732b6d67'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:06.910899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-6cd203af-2d11-4201-8ef0-1c462ae01cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78c926796cc5409a9244540c732b6d67'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:06.922263 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3820201-28e4-4cd1-9291-2003b29bcd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:06.922929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-6cd203af-2d11-4201-8ef0-1c462ae01cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78c926796cc5409a9244540c732b6d67'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:06.923067 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3820201-28e4-4cd1-9291-2003b29bcd60 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:06.923481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3820201-28e4-4cd1-9291-2003b29bcd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:06.923938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3820201-28e4-4cd1-9291-2003b29bcd60 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:06.924755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3820201-28e4-4cd1-9291-2003b29bcd60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:06.926540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3820201-28e4-4cd1-9291-2003b29bcd60 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:06.948567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-6cd203af-2d11-4201-8ef0-1c462ae01cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78c926796cc5409a9244540c732b6d67'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:06.948979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 898/1833] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:06.949958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3820201-28e4-4cd1-9291-2003b29bcd60 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:06.955170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b3820201-28e4-4cd1-9291-2003b29bcd60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:06.955769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-6cd203af-2d11-4201-8ef0-1c462ae01cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78c926796cc5409a9244540c732b6d67'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:07.006800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 936/1834] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:06 2022] DELETE /identity/v3/users/78c926796cc5409a9244540c732b6d67 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:07.006800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd203af-2d11-4201-8ef0-1c462ae01cff admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:07.009153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-6cd203af-2d11-4201-8ef0-1c462ae01cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:07.133687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3820201-28e4-4cd1-9291-2003b29bcd60 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:07.161303 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd203af-2d11-4201-8ef0-1c462ae01cff admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:07.167609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eca8dc-5bbf-43f2-a0c6-71c8df43ab0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:07.169821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eca8dc-5bbf-43f2-a0c6-71c8df43ab0f admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:07.170544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eca8dc-5bbf-43f2-a0c6-71c8df43ab0f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:07.170969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eca8dc-5bbf-43f2-a0c6-71c8df43ab0f admin admin] PATH_INFO: `/v3/projects/1ccf0c7cabe44eebb95f4022a8ec52f9/users/493b563a11f74716a54b826c481ac7d2/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:07.205096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eca8dc-5bbf-43f2-a0c6-71c8df43ab0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ccf0c7cabe44eebb95f4022a8ec52f9, user_id=493b563a11f74716a54b826c481ac7d2, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:07.209220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eca8dc-5bbf-43f2-a0c6-71c8df43ab0f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:07.211464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bcee6-2ab0-468a-8f89-154e007ade7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:07.213271 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bcee6-2ab0-468a-8f89-154e007ade7b service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:07.213691 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bcee6-2ab0-468a-8f89-154e007ade7b service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:07.214077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bcee6-2ab0-468a-8f89-154e007ade7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:07.216775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-335bcee6-2ab0-468a-8f89-154e007ade7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:07.232096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 937/1835] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:07 2022] PUT /identity/v3/projects/1ccf0c7cabe44eebb95f4022a8ec52f9/users/493b563a11f74716a54b826c481ac7d2/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:07.237465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eca8dc-5bbf-43f2-a0c6-71c8df43ab0f admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:07.355652 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eca8dc-5bbf-43f2-a0c6-71c8df43ab0f admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:07.357278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335bcee6-2ab0-468a-8f89-154e007ade7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:07.362062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335bcee6-2ab0-468a-8f89-154e007ade7b service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:07.367647 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e387c-a858-4677-ab44-ea20fa185cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:07.369658 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e387c-a858-4677-ab44-ea20fa185cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:07.369951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e387c-a858-4677-ab44-ea20fa185cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:07.371968 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e387c-a858-4677-ab44-ea20fa185cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:07.375476 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e387c-a858-4677-ab44-ea20fa185cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:07.377446 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e387c-a858-4677-ab44-ea20fa185cc7 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:07.400585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 899/1836] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:07.418031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 938/1837] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:07.422149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bcee6-2ab0-468a-8f89-154e007ade7b service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:07.436540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bcee6-2ab0-468a-8f89-154e007ade7b service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:07.447566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739b10d-d653-4507-8310-821b42facf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:07.448473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739b10d-d653-4507-8310-821b42facf6a admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:07.448761 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739b10d-d653-4507-8310-821b42facf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:07.449044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739b10d-d653-4507-8310-821b42facf6a admin admin] PATH_INFO: `/v3/projects/1ccf0c7cabe44eebb95f4022a8ec52f9/users/493b563a11f74716a54b826c481ac7d2/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:07.453768 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739b10d-d653-4507-8310-821b42facf6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ccf0c7cabe44eebb95f4022a8ec52f9, user_id=493b563a11f74716a54b826c481ac7d2, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:07.459108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739b10d-d653-4507-8310-821b42facf6a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:07.492186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 900/1838] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:07 2022] PUT /identity/v3/projects/1ccf0c7cabe44eebb95f4022a8ec52f9/users/493b563a11f74716a54b826c481ac7d2/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:07.502795 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ad46f-e380-4897-8927-407cc9b47daf None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:07.503473 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ad46f-e380-4897-8927-407cc9b47daf None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:07.504046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ad46f-e380-4897-8927-407cc9b47daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:07.575427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-668ad46f-e380-4897-8927-407cc9b47daf None None] MFA Rules not processed for user `493b563a11f74716a54b826c481ac7d2`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:07.577931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-668ad46f-e380-4897-8927-407cc9b47daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:07.753793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 939/1839] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:07.764443 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739b10d-d653-4507-8310-821b42facf6a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:07.766560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-8739b10d-d653-4507-8310-821b42facf6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:07.897176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739b10d-d653-4507-8310-821b42facf6a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:07.903751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8a2b9-1bcf-4508-9096-ed4dbe6672d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:07.906251 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8a2b9-1bcf-4508-9096-ed4dbe6672d2 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:07.906542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8a2b9-1bcf-4508-9096-ed4dbe6672d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:07.906823 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8a2b9-1bcf-4508-9096-ed4dbe6672d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:07.925411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8a2b9-1bcf-4508-9096-ed4dbe6672d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:07.926870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8a2b9-1bcf-4508-9096-ed4dbe6672d2 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:07.970478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 901/1840] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:08.673059 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46bb33-51d9-4717-9d90-a181a34e37ed None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:08.673692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46bb33-51d9-4717-9d90-a181a34e37ed None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:08.674184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46bb33-51d9-4717-9d90-a181a34e37ed None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:08.676115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 940/1841] 10.222.0.25 () {54 vars in 921 bytes} [Mon May 9 18:30:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:30:08.697077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639614e-248f-4e03-953f-d6de0c53f515 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:08.700234 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639614e-248f-4e03-953f-d6de0c53f515 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:08.700582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639614e-248f-4e03-953f-d6de0c53f515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:08.727693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46bb33-51d9-4717-9d90-a181a34e37ed None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:08.733918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-cf46bb33-51d9-4717-9d90-a181a34e37ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:08.766081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-b639614e-248f-4e03-953f-d6de0c53f515 None None] MFA Rules not processed for user `f769ee9f31354e5296645fb1e3bd84b6`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:08.769431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b639614e-248f-4e03-953f-d6de0c53f515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:08.827369 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46bb33-51d9-4717-9d90-a181a34e37ed None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:08.839345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:08.840070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:08.840567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:08.840936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] PATH_INFO: `/v3/projects/3f33e0f1501e4949b413c59dbbeaf24a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:08.844567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f33e0f1501e4949b413c59dbbeaf24a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:08.846599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:08.976167 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f33e0f1501e4949b413c59dbbeaf24a', 'request_id': 'req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:08.981595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 902/1842] 10.222.0.25 () {58 vars in 1018 bytes} [Mon May 9 18:30:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:08.982853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639614e-248f-4e03-953f-d6de0c53f515 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:08.991979 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f33e0f1501e4949b413c59dbbeaf24a', 'request_id': 'req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:09.010684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f33e0f1501e4949b413c59dbbeaf24a', 'request_id': 'req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:09.012098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639614e-248f-4e03-953f-d6de0c53f515 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:09.034117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] The token cache is being invalidate because project 3f33e0f1501e4949b413c59dbbeaf24a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:09.035373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:09.043307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:09.046838 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 941/1843] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:08 2022] DELETE /identity/v3/projects/3f33e0f1501e4949b413c59dbbeaf24a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:09.047869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:09.055514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:09.111031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec4db4-e27e-4b63-b8c5-7615543fce10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:09.111763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec4db4-e27e-4b63-b8c5-7615543fce10 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:09.112051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec4db4-e27e-4b63-b8c5-7615543fce10 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:09.112326 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec4db4-e27e-4b63-b8c5-7615543fce10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:09.115090 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f4ec4db4-e27e-4b63-b8c5-7615543fce10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:09.205196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c945b2-4e40-40f6-aefd-66cf480e8e4e admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:09.220495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec4db4-e27e-4b63-b8c5-7615543fce10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:09.222096 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec4db4-e27e-4b63-b8c5-7615543fce10 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:09.224320 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c6206-13cd-43fa-8736-86b5dde1377e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:09.226065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c6206-13cd-43fa-8736-86b5dde1377e service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:09.226581 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c6206-13cd-43fa-8736-86b5dde1377e service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:09.226874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c6206-13cd-43fa-8736-86b5dde1377e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:09.232334 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-481c6206-13cd-43fa-8736-86b5dde1377e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:09.247138 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 903/1844] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:09.249197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c6206-13cd-43fa-8736-86b5dde1377e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:09.249417 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393f302-fa5e-4432-aacb-d4b03df534c7 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:09.249721 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393f302-fa5e-4432-aacb-d4b03df534c7 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:09.250795 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c6206-13cd-43fa-8736-86b5dde1377e service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:09.251608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393f302-fa5e-4432-aacb-d4b03df534c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:09.262442 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 942/1845] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:09.263410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c6206-13cd-43fa-8736-86b5dde1377e service nova] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:09.265072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-481c6206-13cd-43fa-8736-86b5dde1377e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:09.280310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c6206-13cd-43fa-8736-86b5dde1377e service nova] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:09.291393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156fb71-dc52-4b94-a2e8-62c43778581e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:09.292068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156fb71-dc52-4b94-a2e8-62c43778581e service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:09.292355 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156fb71-dc52-4b94-a2e8-62c43778581e service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:09.292824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156fb71-dc52-4b94-a2e8-62c43778581e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:09.296600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4156fb71-dc52-4b94-a2e8-62c43778581e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:09.343357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-7393f302-fa5e-4432-aacb-d4b03df534c7 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:09.344808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7393f302-fa5e-4432-aacb-d4b03df534c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:09.453624 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156fb71-dc52-4b94-a2e8-62c43778581e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:09.455784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156fb71-dc52-4b94-a2e8-62c43778581e service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:09.486475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 943/1846] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:09 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:09.487989 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92fd1d-9774-4fee-ac57-d5a2c114ecfa None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:09.488282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92fd1d-9774-4fee-ac57-d5a2c114ecfa None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:09.488655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92fd1d-9774-4fee-ac57-d5a2c114ecfa None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:09.490038 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 944/1847] 10.222.0.25 () {54 vars in 921 bytes} [Mon May 9 18:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:30:09.507464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5332434-a676-425e-8d7b-228bd65ee35e None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:09.507986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5332434-a676-425e-8d7b-228bd65ee35e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:09.508407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5332434-a676-425e-8d7b-228bd65ee35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:09.545731 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 904/1848] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:09.547473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380edda3-7c95-4e87-b6dc-eee15a58833c None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:09.547763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380edda3-7c95-4e87-b6dc-eee15a58833c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:09.548043 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380edda3-7c95-4e87-b6dc-eee15a58833c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:09.566260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c5332434-a676-425e-8d7b-228bd65ee35e None None] MFA Rules not processed for user `f769ee9f31354e5296645fb1e3bd84b6`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:09.567502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c5332434-a676-425e-8d7b-228bd65ee35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:09.592601 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-380edda3-7c95-4e87-b6dc-eee15a58833c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:09.594347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-380edda3-7c95-4e87-b6dc-eee15a58833c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:09.606729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 905/1849] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:09.610231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 945/1850] 10.222.0.25 () {58 vars in 1018 bytes} [Mon May 9 18:30:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:09.611561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5332434-a676-425e-8d7b-228bd65ee35e None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:09.638374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5332434-a676-425e-8d7b-228bd65ee35e None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:09.653794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2a51a-30b8-438c-8dab-8454774d2682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:09.655428 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2a51a-30b8-438c-8dab-8454774d2682 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:09.661611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2a51a-30b8-438c-8dab-8454774d2682 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:09.661986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2a51a-30b8-438c-8dab-8454774d2682 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:09.666061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2a51a-30b8-438c-8dab-8454774d2682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:30:09.666292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2a51a-30b8-438c-8dab-8454774d2682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:09.683745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2a51a-30b8-438c-8dab-8454774d2682 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:09.744219 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 946/1851] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:30:09.749383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380edda3-7c95-4e87-b6dc-eee15a58833c None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:09.761322 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380edda3-7c95-4e87-b6dc-eee15a58833c None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:09.773708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f50d49-0096-4cdd-8709-e0a09c7e4af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:09.774395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f50d49-0096-4cdd-8709-e0a09c7e4af8 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:09.782725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f50d49-0096-4cdd-8709-e0a09c7e4af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:09.783315 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f50d49-0096-4cdd-8709-e0a09c7e4af8 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:09.784324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f50d49-0096-4cdd-8709-e0a09c7e4af8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:09.788691 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f50d49-0096-4cdd-8709-e0a09c7e4af8 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:09.857530 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 906/1852] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:09 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:09.862008 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2a51a-30b8-438c-8dab-8454774d2682 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:09.890705 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2a51a-30b8-438c-8dab-8454774d2682 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:10.015616 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58744a21-2fca-45e3-9690-2b2042af8faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:10.017100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58744a21-2fca-45e3-9690-2b2042af8faf admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:10.017807 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58744a21-2fca-45e3-9690-2b2042af8faf admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:10.019949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58744a21-2fca-45e3-9690-2b2042af8faf admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:10.020494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58744a21-2fca-45e3-9690-2b2042af8faf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:10.023034 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58744a21-2fca-45e3-9690-2b2042af8faf admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:10.065741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 947/1853] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:09 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:10.069702 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f50d49-0096-4cdd-8709-e0a09c7e4af8 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:10.084128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f50d49-0096-4cdd-8709-e0a09c7e4af8 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:10.092332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c1f48-33b6-44d6-95c4-437cdade37aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:10.093300 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c1f48-33b6-44d6-95c4-437cdade37aa admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:10.093695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c1f48-33b6-44d6-95c4-437cdade37aa admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:10.093978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c1f48-33b6-44d6-95c4-437cdade37aa admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:10.094898 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c1f48-33b6-44d6-95c4-437cdade37aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:10.097682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c1f48-33b6-44d6-95c4-437cdade37aa admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:10.115831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 907/1854] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:10.119782 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58744a21-2fca-45e3-9690-2b2042af8faf admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:10.131877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58744a21-2fca-45e3-9690-2b2042af8faf admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:10.136747 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e8a82-9255-4ef4-ab35-7c3915b2093a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:10.137422 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e8a82-9255-4ef4-ab35-7c3915b2093a admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:10.137703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e8a82-9255-4ef4-ab35-7c3915b2093a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:10.137983 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e8a82-9255-4ef4-ab35-7c3915b2093a admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:10.138750 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e8a82-9255-4ef4-ab35-7c3915b2093a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:10.140374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e8a82-9255-4ef4-ab35-7c3915b2093a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:10.150708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 948/1855] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:10.154829 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c1f48-33b6-44d6-95c4-437cdade37aa admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:10.164939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c1f48-33b6-44d6-95c4-437cdade37aa admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:10.170980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608dcfd-63f8-494b-ac8e-a9b5544c9173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:10.171994 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608dcfd-63f8-494b-ac8e-a9b5544c9173 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:10.172349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608dcfd-63f8-494b-ac8e-a9b5544c9173 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:10.172720 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608dcfd-63f8-494b-ac8e-a9b5544c9173 admin admin] PATH_INFO: `/v3/projects/e5d3e00e3fd7464e9c7f2e44fd6bc288/users/5f5f1e1a038a46db932a8a1956ad764e/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:10.191915 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608dcfd-63f8-494b-ac8e-a9b5544c9173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d3e00e3fd7464e9c7f2e44fd6bc288, user_id=5f5f1e1a038a46db932a8a1956ad764e, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:10.194928 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608dcfd-63f8-494b-ac8e-a9b5544c9173 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:10.228389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b9f31-e40c-47ef-b840-f6a980bc33fc None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:10.228539 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b9f31-e40c-47ef-b840-f6a980bc33fc None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:10.228915 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b9f31-e40c-47ef-b840-f6a980bc33fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:10.230533 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 908/1856] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:10 2022] PUT /identity/v3/projects/e5d3e00e3fd7464e9c7f2e44fd6bc288/users/5f5f1e1a038a46db932a8a1956ad764e/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:10.262530 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608dcfd-63f8-494b-ac8e-a9b5544c9173 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:10.291954 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-849b9f31-e40c-47ef-b840-f6a980bc33fc None None] MFA Rules not processed for user `5f5f1e1a038a46db932a8a1956ad764e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:10.293249 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-849b9f31-e40c-47ef-b840-f6a980bc33fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:10.407211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608dcfd-63f8-494b-ac8e-a9b5544c9173 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:10.426468 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ce4f1-f061-4e54-8538-e84c1b228dfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:10.429065 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ce4f1-f061-4e54-8538-e84c1b228dfc service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:10.429271 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ce4f1-f061-4e54-8538-e84c1b228dfc service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:10.429554 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ce4f1-f061-4e54-8538-e84c1b228dfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:10.460528 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 949/1857] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:10.461341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ce4f1-f061-4e54-8538-e84c1b228dfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:10.461901 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b9f31-e40c-47ef-b840-f6a980bc33fc None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:10.465907 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ce4f1-f061-4e54-8538-e84c1b228dfc service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:10.494320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 909/1858] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:10.631723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b9f31-e40c-47ef-b840-f6a980bc33fc None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:10.639399 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee0511-8d1a-4d98-8d6e-731abe4c303a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:10.640270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee0511-8d1a-4d98-8d6e-731abe4c303a admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:10.640654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee0511-8d1a-4d98-8d6e-731abe4c303a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:10.641475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee0511-8d1a-4d98-8d6e-731abe4c303a admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:10.642711 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee0511-8d1a-4d98-8d6e-731abe4c303a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:10.648775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee0511-8d1a-4d98-8d6e-731abe4c303a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:10.700251 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 950/1859] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:10 2022] POST /identity/v3/projects => generated 383 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:10.703218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ce4f1-f061-4e54-8538-e84c1b228dfc service nova] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:10.725122 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ce4f1-f061-4e54-8538-e84c1b228dfc service nova] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:10.849153 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f07b1-82fd-4855-be77-5d66ac35be62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:10.849904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f07b1-82fd-4855-be77-5d66ac35be62 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:10.850409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f07b1-82fd-4855-be77-5d66ac35be62 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:10.850863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f07b1-82fd-4855-be77-5d66ac35be62 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:10.851902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f07b1-82fd-4855-be77-5d66ac35be62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:10.854301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f07b1-82fd-4855-be77-5d66ac35be62 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:10.893876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee0511-8d1a-4d98-8d6e-731abe4c303a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:10.904065 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 910/1860] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:10 2022] POST /identity/v3/users => generated 298 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:10.916217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee0511-8d1a-4d98-8d6e-731abe4c303a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:10.926592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80dd06-2dc2-47d8-b867-0621c4ece25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:10.930580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80dd06-2dc2-47d8-b867-0621c4ece25d admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:10.931012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80dd06-2dc2-47d8-b867-0621c4ece25d admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:10.931490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80dd06-2dc2-47d8-b867-0621c4ece25d admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:10.934353 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80dd06-2dc2-47d8-b867-0621c4ece25d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:10.940574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80dd06-2dc2-47d8-b867-0621c4ece25d admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:10.976698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 951/1861] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:10.981562 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f07b1-82fd-4855-be77-5d66ac35be62 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:10.993315 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f07b1-82fd-4855-be77-5d66ac35be62 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:10.996041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80dd06-2dc2-47d8-b867-0621c4ece25d admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:10.998639 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-db80dd06-2dc2-47d8-b867-0621c4ece25d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:11.010786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf44139-26cf-4ebd-940a-59306ce10bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:11.011752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf44139-26cf-4ebd-940a-59306ce10bb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:11.012032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf44139-26cf-4ebd-940a-59306ce10bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:11.012302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf44139-26cf-4ebd-940a-59306ce10bb7 admin admin] PATH_INFO: `/v3/projects/aa5f55a14cd8438e898de970686597f6/users/370d515a06ec468ea437e3b31a556dcb/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:11.028999 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80dd06-2dc2-47d8-b867-0621c4ece25d admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:11.039721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda4b4e-463b-4cd4-a6a6-8f08a5b5cb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:11.041042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda4b4e-463b-4cd4-a6a6-8f08a5b5cb43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:11.042081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf44139-26cf-4ebd-940a-59306ce10bb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa5f55a14cd8438e898de970686597f6, user_id=370d515a06ec468ea437e3b31a556dcb, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:11.048378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf44139-26cf-4ebd-940a-59306ce10bb7 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:11.053732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda4b4e-463b-4cd4-a6a6-8f08a5b5cb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:11.055093 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda4b4e-463b-4cd4-a6a6-8f08a5b5cb43 admin admin] PATH_INFO: `/v3/users/1ac317b9a0ea4aca84c20c39c5896a89` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:11.064244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda4b4e-463b-4cd4-a6a6-8f08a5b5cb43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ac317b9a0ea4aca84c20c39c5896a89)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:11.067356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda4b4e-463b-4cd4-a6a6-8f08a5b5cb43 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:11.078024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 911/1862] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:10 2022] PUT /identity/v3/projects/aa5f55a14cd8438e898de970686597f6/users/370d515a06ec468ea437e3b31a556dcb/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:11.079922 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf44139-26cf-4ebd-940a-59306ce10bb7 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:11.256455 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf44139-26cf-4ebd-940a-59306ce10bb7 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:11.267111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28342037-e92c-4a80-91dc-a6725acfa7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:11.267844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28342037-e92c-4a80-91dc-a6725acfa7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:11.268135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28342037-e92c-4a80-91dc-a6725acfa7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:11.268650 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28342037-e92c-4a80-91dc-a6725acfa7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:11.269401 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28342037-e92c-4a80-91dc-a6725acfa7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:11.271991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28342037-e92c-4a80-91dc-a6725acfa7c3 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:11.288997 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 912/1863] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:11.293020 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28342037-e92c-4a80-91dc-a6725acfa7c3 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:11.312854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28342037-e92c-4a80-91dc-a6725acfa7c3 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:11.322106 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38506a-f496-43dc-9f5f-491eca52cd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:11.322106 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38506a-f496-43dc-9f5f-491eca52cd57 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:11.322385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38506a-f496-43dc-9f5f-491eca52cd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:11.322879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38506a-f496-43dc-9f5f-491eca52cd57 admin admin] PATH_INFO: `/v3/projects/aa5f55a14cd8438e898de970686597f6/users/370d515a06ec468ea437e3b31a556dcb/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:11.326513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38506a-f496-43dc-9f5f-491eca52cd57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa5f55a14cd8438e898de970686597f6, user_id=370d515a06ec468ea437e3b31a556dcb, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:11.330200 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38506a-f496-43dc-9f5f-491eca52cd57 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:11.349571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 913/1864] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:11 2022] PUT /identity/v3/projects/aa5f55a14cd8438e898de970686597f6/users/370d515a06ec468ea437e3b31a556dcb/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:11.365179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae3ecf-93e6-482c-a777-567c5981bc98 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:11.365569 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae3ecf-93e6-482c-a777-567c5981bc98 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:11.365972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae3ecf-93e6-482c-a777-567c5981bc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:11.455685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-28ae3ecf-93e6-482c-a777-567c5981bc98 None None] MFA Rules not processed for user `370d515a06ec468ea437e3b31a556dcb`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:11.457055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-28ae3ecf-93e6-482c-a777-567c5981bc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:11.486778 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3dda4b4e-463b-4cd4-a6a6-8f08a5b5cb43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ac317b9a0ea4aca84c20c39c5896a89'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:11.489130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3dda4b4e-463b-4cd4-a6a6-8f08a5b5cb43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ac317b9a0ea4aca84c20c39c5896a89'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:11.500451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3dda4b4e-463b-4cd4-a6a6-8f08a5b5cb43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ac317b9a0ea4aca84c20c39c5896a89'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:11.516795 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3dda4b4e-463b-4cd4-a6a6-8f08a5b5cb43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ac317b9a0ea4aca84c20c39c5896a89'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:11.519795 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3dda4b4e-463b-4cd4-a6a6-8f08a5b5cb43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ac317b9a0ea4aca84c20c39c5896a89'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:11.558018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 952/1865] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:10 2022] DELETE /identity/v3/users/1ac317b9a0ea4aca84c20c39c5896a89 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:11.563294 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2cad2-dd0f-4580-903d-3be5165446aa None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:11.563788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2cad2-dd0f-4580-903d-3be5165446aa None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:11.564155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2cad2-dd0f-4580-903d-3be5165446aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:11.587506 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-47f2cad2-dd0f-4580-903d-3be5165446aa None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:11.589335 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-47f2cad2-dd0f-4580-903d-3be5165446aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:11.606347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 914/1866] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:11.617627 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae3ecf-93e6-482c-a777-567c5981bc98 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:11.622063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-28ae3ecf-93e6-482c-a777-567c5981bc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:11.709258 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 953/1867] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:11.715622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2cad2-dd0f-4580-903d-3be5165446aa None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:11.717807 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-47f2cad2-dd0f-4580-903d-3be5165446aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:11.742489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae3ecf-93e6-482c-a777-567c5981bc98 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:11.794113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2cad2-dd0f-4580-903d-3be5165446aa None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:11.801329 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57f0ca-60e7-4f1a-bafd-460ba1b1d61f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:11.801986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57f0ca-60e7-4f1a-bafd-460ba1b1d61f service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:11.802292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57f0ca-60e7-4f1a-bafd-460ba1b1d61f service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:11.802552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57f0ca-60e7-4f1a-bafd-460ba1b1d61f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:11.812671 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24ea61-1f9d-47af-bb2c-662ef7598ecf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:11.813762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24ea61-1f9d-47af-bb2c-662ef7598ecf service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:11.814246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24ea61-1f9d-47af-bb2c-662ef7598ecf service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:11.814884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24ea61-1f9d-47af-bb2c-662ef7598ecf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:11.818039 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57f0ca-60e7-4f1a-bafd-460ba1b1d61f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:11.819525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57f0ca-60e7-4f1a-bafd-460ba1b1d61f service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:11.830733 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 954/1868] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:11.915489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24ea61-1f9d-47af-bb2c-662ef7598ecf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:11.919157 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24ea61-1f9d-47af-bb2c-662ef7598ecf service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:11.961202 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 915/1869] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:12.092831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57f0ca-60e7-4f1a-bafd-460ba1b1d61f service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:12.106442 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57f0ca-60e7-4f1a-bafd-460ba1b1d61f service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:12.112501 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6de617-e5f6-4b3f-8f3d-50f800c5f8ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:12.113408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6de617-e5f6-4b3f-8f3d-50f800c5f8ae service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:12.113893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6de617-e5f6-4b3f-8f3d-50f800c5f8ae service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:12.114473 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6de617-e5f6-4b3f-8f3d-50f800c5f8ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:12.117163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-0d6de617-e5f6-4b3f-8f3d-50f800c5f8ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:12.209106 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6de617-e5f6-4b3f-8f3d-50f800c5f8ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:12.210963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6de617-e5f6-4b3f-8f3d-50f800c5f8ae service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:12.234102 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 955/1870] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:13.278069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24ea61-1f9d-47af-bb2c-662ef7598ecf service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:13.281194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9c24ea61-1f9d-47af-bb2c-662ef7598ecf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:13.321458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24ea61-1f9d-47af-bb2c-662ef7598ecf service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:13.339451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e3c3f-a036-4e06-8d27-800047bd8a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:13.340772 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e3c3f-a036-4e06-8d27-800047bd8a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:13.341097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e3c3f-a036-4e06-8d27-800047bd8a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:13.341380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e3c3f-a036-4e06-8d27-800047bd8a7b admin admin] PATH_INFO: `/v3/projects/36bd9f2168e24ceda5308fe4919649be` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:13.345693 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e3c3f-a036-4e06-8d27-800047bd8a7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=36bd9f2168e24ceda5308fe4919649be)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:13.355781 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e3c3f-a036-4e06-8d27-800047bd8a7b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:13.502895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a83e3c3f-a036-4e06-8d27-800047bd8a7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36bd9f2168e24ceda5308fe4919649be', 'request_id': 'req-a83e3c3f-a036-4e06-8d27-800047bd8a7b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:13.518185 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a83e3c3f-a036-4e06-8d27-800047bd8a7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36bd9f2168e24ceda5308fe4919649be', 'request_id': 'req-a83e3c3f-a036-4e06-8d27-800047bd8a7b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:13.532742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a83e3c3f-a036-4e06-8d27-800047bd8a7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36bd9f2168e24ceda5308fe4919649be', 'request_id': 'req-a83e3c3f-a036-4e06-8d27-800047bd8a7b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:13.552688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a83e3c3f-a036-4e06-8d27-800047bd8a7b admin admin] The token cache is being invalidate because project 36bd9f2168e24ceda5308fe4919649be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:13.552991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a83e3c3f-a036-4e06-8d27-800047bd8a7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:13.556640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a83e3c3f-a036-4e06-8d27-800047bd8a7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:13.559695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 916/1871] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:13 2022] DELETE /identity/v3/projects/36bd9f2168e24ceda5308fe4919649be => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:13.573060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b15fb7-6e7a-4b9d-9088-3900ec6b6fff None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:13.573503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b15fb7-6e7a-4b9d-9088-3900ec6b6fff None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:13.573608 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b15fb7-6e7a-4b9d-9088-3900ec6b6fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:13.597667 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b4b15fb7-6e7a-4b9d-9088-3900ec6b6fff None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:13.599067 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b4b15fb7-6e7a-4b9d-9088-3900ec6b6fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:13.691666 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 956/1872] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:13.699585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4732fdb-b93a-4d5b-abf4-5fd53cf87d4d None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:13.699967 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4732fdb-b93a-4d5b-abf4-5fd53cf87d4d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:13.700303 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4732fdb-b93a-4d5b-abf4-5fd53cf87d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:13.719453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-c4732fdb-b93a-4d5b-abf4-5fd53cf87d4d None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:13.720638 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c4732fdb-b93a-4d5b-abf4-5fd53cf87d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:13.729123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 917/1873] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:13.733794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b15fb7-6e7a-4b9d-9088-3900ec6b6fff None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:13.743732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b15fb7-6e7a-4b9d-9088-3900ec6b6fff None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:13.753055 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcd2fe-5a74-455e-b4c0-efa42bcaf0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:13.754351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcd2fe-5a74-455e-b4c0-efa42bcaf0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:13.754351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcd2fe-5a74-455e-b4c0-efa42bcaf0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:13.754351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcd2fe-5a74-455e-b4c0-efa42bcaf0f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:13.755234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcd2fe-5a74-455e-b4c0-efa42bcaf0f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:30:13.755605 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcd2fe-5a74-455e-b4c0-efa42bcaf0f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:13.757425 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcd2fe-5a74-455e-b4c0-efa42bcaf0f7 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:13.771881 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 957/1874] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:30:13.776227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4732fdb-b93a-4d5b-abf4-5fd53cf87d4d None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:13.786194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4732fdb-b93a-4d5b-abf4-5fd53cf87d4d None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:13.791058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f910e5-2b1a-4cc4-9c9b-1ec69a38140c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:13.791757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f910e5-2b1a-4cc4-9c9b-1ec69a38140c admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:13.792041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f910e5-2b1a-4cc4-9c9b-1ec69a38140c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:13.792318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f910e5-2b1a-4cc4-9c9b-1ec69a38140c admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:13.793039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f910e5-2b1a-4cc4-9c9b-1ec69a38140c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:13.795741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f910e5-2b1a-4cc4-9c9b-1ec69a38140c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:13.824163 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 918/1875] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:13 2022] POST /identity/v3/projects => generated 403 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:13.825712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcd2fe-5a74-455e-b4c0-efa42bcaf0f7 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:13.837634 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcd2fe-5a74-455e-b4c0-efa42bcaf0f7 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:13.908187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056b5d1-3634-496c-820e-67f3f09899a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:13.908899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056b5d1-3634-496c-820e-67f3f09899a4 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:13.909184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056b5d1-3634-496c-820e-67f3f09899a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:13.909462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056b5d1-3634-496c-820e-67f3f09899a4 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:13.910190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056b5d1-3634-496c-820e-67f3f09899a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:13.912219 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056b5d1-3634-496c-820e-67f3f09899a4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:13.930878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 958/1876] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:13 2022] POST /identity/v3/users => generated 302 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:13.935352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f910e5-2b1a-4cc4-9c9b-1ec69a38140c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:13.945436 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f910e5-2b1a-4cc4-9c9b-1ec69a38140c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:13.950850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7ac9b-850a-4c12-ac73-f37d8425fad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:13.952090 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7ac9b-850a-4c12-ac73-f37d8425fad6 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:13.952393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7ac9b-850a-4c12-ac73-f37d8425fad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:13.952768 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7ac9b-850a-4c12-ac73-f37d8425fad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:13.953496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7ac9b-850a-4c12-ac73-f37d8425fad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:13.955077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7ac9b-850a-4c12-ac73-f37d8425fad6 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:13.966059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 919/1877] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:13.970670 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056b5d1-3634-496c-820e-67f3f09899a4 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:13.980979 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056b5d1-3634-496c-820e-67f3f09899a4 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:13.985697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8952dd2-a61e-47e3-bfa8-09601f502682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:13.986414 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8952dd2-a61e-47e3-bfa8-09601f502682 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:13.986699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8952dd2-a61e-47e3-bfa8-09601f502682 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:13.986986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8952dd2-a61e-47e3-bfa8-09601f502682 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:13.987815 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8952dd2-a61e-47e3-bfa8-09601f502682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:13.989515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8952dd2-a61e-47e3-bfa8-09601f502682 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:14.000781 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 959/1878] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:14.005394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7ac9b-850a-4c12-ac73-f37d8425fad6 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:14.016114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7ac9b-850a-4c12-ac73-f37d8425fad6 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:14.020965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa84a9-e317-45ec-b31e-ed9b39fae259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:14.021857 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa84a9-e317-45ec-b31e-ed9b39fae259 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:14.022167 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa84a9-e317-45ec-b31e-ed9b39fae259 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:14.022450 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa84a9-e317-45ec-b31e-ed9b39fae259 admin admin] PATH_INFO: `/v3/projects/41f92d3057864904aa809c178c99789a/users/7155dd4b7f684400a85f02709e774d7f/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:14.037712 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa84a9-e317-45ec-b31e-ed9b39fae259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f92d3057864904aa809c178c99789a, user_id=7155dd4b7f684400a85f02709e774d7f, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:14.041081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa84a9-e317-45ec-b31e-ed9b39fae259 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:14.055170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 920/1879] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:14 2022] PUT /identity/v3/projects/41f92d3057864904aa809c178c99789a/users/7155dd4b7f684400a85f02709e774d7f/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:14.060227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a75a94-cb4f-4e59-a084-2304d97f1a40 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:14.060517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a75a94-cb4f-4e59-a084-2304d97f1a40 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:14.060795 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a75a94-cb4f-4e59-a084-2304d97f1a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:14.078964 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-50a75a94-cb4f-4e59-a084-2304d97f1a40 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:14.080292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-50a75a94-cb4f-4e59-a084-2304d97f1a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:14.142375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 960/1880] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:14.151842 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa84a9-e317-45ec-b31e-ed9b39fae259 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:14.153513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-5dfa84a9-e317-45ec-b31e-ed9b39fae259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:14.212597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa84a9-e317-45ec-b31e-ed9b39fae259 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:14.216590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657f077-a692-417a-98f6-4044f74ff1f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:14.217247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657f077-a692-417a-98f6-4044f74ff1f8 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:14.217533 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657f077-a692-417a-98f6-4044f74ff1f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:14.217813 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657f077-a692-417a-98f6-4044f74ff1f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:14.226951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657f077-a692-417a-98f6-4044f74ff1f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:14.228505 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657f077-a692-417a-98f6-4044f74ff1f8 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:14.238186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 921/1881] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:30:14.629810 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a75a94-cb4f-4e59-a084-2304d97f1a40 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:14.631625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-50a75a94-cb4f-4e59-a084-2304d97f1a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:14.775647 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a75a94-cb4f-4e59-a084-2304d97f1a40 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:14.780473 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f681b88-e967-4080-8d75-a90942309cc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:14.781547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f681b88-e967-4080-8d75-a90942309cc7 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:14.782212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f681b88-e967-4080-8d75-a90942309cc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:14.782759 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f681b88-e967-4080-8d75-a90942309cc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:14.785035 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4f681b88-e967-4080-8d75-a90942309cc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:14.871794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f681b88-e967-4080-8d75-a90942309cc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:14.874116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f681b88-e967-4080-8d75-a90942309cc7 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:14.900305 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 961/1882] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:15.288477 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657f077-a692-417a-98f6-4044f74ff1f8 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:15.290182 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1657f077-a692-417a-98f6-4044f74ff1f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:15.306122 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657f077-a692-417a-98f6-4044f74ff1f8 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:15.312255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4d031-4d18-4bca-a227-99f70f8aceda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:15.313535 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4d031-4d18-4bca-a227-99f70f8aceda admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:15.314031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4d031-4d18-4bca-a227-99f70f8aceda admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:15.314505 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4d031-4d18-4bca-a227-99f70f8aceda admin admin] PATH_INFO: `/v3/users/5f5f1e1a038a46db932a8a1956ad764e` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:15.316013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4d031-4d18-4bca-a227-99f70f8aceda admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f5f1e1a038a46db932a8a1956ad764e)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:15.318467 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4d031-4d18-4bca-a227-99f70f8aceda admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:15.387695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-fae4d031-4d18-4bca-a227-99f70f8aceda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f5f1e1a038a46db932a8a1956ad764e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:15.395514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-fae4d031-4d18-4bca-a227-99f70f8aceda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f5f1e1a038a46db932a8a1956ad764e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:15.396413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-fae4d031-4d18-4bca-a227-99f70f8aceda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f5f1e1a038a46db932a8a1956ad764e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:15.405092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-fae4d031-4d18-4bca-a227-99f70f8aceda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f5f1e1a038a46db932a8a1956ad764e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:15.405883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-fae4d031-4d18-4bca-a227-99f70f8aceda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f5f1e1a038a46db932a8a1956ad764e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:15.424169 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 922/1883] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:15 2022] DELETE /identity/v3/users/5f5f1e1a038a46db932a8a1956ad764e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:15.429261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776963ae-1e19-4c39-b22c-3ed3d1d121e8 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:15.429636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776963ae-1e19-4c39-b22c-3ed3d1d121e8 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:15.429924 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776963ae-1e19-4c39-b22c-3ed3d1d121e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:15.457791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-776963ae-1e19-4c39-b22c-3ed3d1d121e8 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:15.460578 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-776963ae-1e19-4c39-b22c-3ed3d1d121e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:15.553019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 962/1884] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:15.567146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4d031-4d18-4bca-a227-99f70f8aceda admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:15.569538 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-fae4d031-4d18-4bca-a227-99f70f8aceda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:15.639838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4d031-4d18-4bca-a227-99f70f8aceda admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:15.645068 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd3d55-2255-4be5-926d-e0a67f3baf81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:15.645710 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd3d55-2255-4be5-926d-e0a67f3baf81 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:15.645995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd3d55-2255-4be5-926d-e0a67f3baf81 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:15.646276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd3d55-2255-4be5-926d-e0a67f3baf81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:15.663533 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd3d55-2255-4be5-926d-e0a67f3baf81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:15.665240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd3d55-2255-4be5-926d-e0a67f3baf81 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:15.685897 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 923/1885] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:16.009631 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776963ae-1e19-4c39-b22c-3ed3d1d121e8 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:16.011885 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-776963ae-1e19-4c39-b22c-3ed3d1d121e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:16.029893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776963ae-1e19-4c39-b22c-3ed3d1d121e8 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:16.035763 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f925cf-f039-4633-bbaf-397a051d580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:16.037251 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f925cf-f039-4633-bbaf-397a051d580e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:16.037853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f925cf-f039-4633-bbaf-397a051d580e admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:16.038351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f925cf-f039-4633-bbaf-397a051d580e admin admin] PATH_INFO: `/v3/projects/e5d3e00e3fd7464e9c7f2e44fd6bc288` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:16.040332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f925cf-f039-4633-bbaf-397a051d580e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d3e00e3fd7464e9c7f2e44fd6bc288)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:16.043208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f925cf-f039-4633-bbaf-397a051d580e admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:16.106126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-42f925cf-f039-4633-bbaf-397a051d580e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d3e00e3fd7464e9c7f2e44fd6bc288', 'request_id': 'req-42f925cf-f039-4633-bbaf-397a051d580e'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:16.120335 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-42f925cf-f039-4633-bbaf-397a051d580e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d3e00e3fd7464e9c7f2e44fd6bc288', 'request_id': 'req-42f925cf-f039-4633-bbaf-397a051d580e'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:16.136602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-42f925cf-f039-4633-bbaf-397a051d580e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d3e00e3fd7464e9c7f2e44fd6bc288', 'request_id': 'req-42f925cf-f039-4633-bbaf-397a051d580e'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:16.160271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-42f925cf-f039-4633-bbaf-397a051d580e admin admin] The token cache is being invalidate because project e5d3e00e3fd7464e9c7f2e44fd6bc288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:16.160271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-42f925cf-f039-4633-bbaf-397a051d580e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:16.160271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-42f925cf-f039-4633-bbaf-397a051d580e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:16.164017 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 963/1886] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:16 2022] DELETE /identity/v3/projects/e5d3e00e3fd7464e9c7f2e44fd6bc288 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:16.167302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd3d55-2255-4be5-926d-e0a67f3baf81 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:16.191506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd3d55-2255-4be5-926d-e0a67f3baf81 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:16.291853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07b913-a484-4466-bac3-bab9eb422ef8 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:16.292141 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07b913-a484-4466-bac3-bab9eb422ef8 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:16.292748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07b913-a484-4466-bac3-bab9eb422ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:16.321210 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299439a-4ca0-4b61-b2bd-4a6bfca60d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:16.322217 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299439a-4ca0-4b61-b2bd-4a6bfca60d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:16.322513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299439a-4ca0-4b61-b2bd-4a6bfca60d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:16.322940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299439a-4ca0-4b61-b2bd-4a6bfca60d54 admin admin] PATH_INFO: `/v3/users/370d515a06ec468ea437e3b31a556dcb` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:16.328840 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299439a-4ca0-4b61-b2bd-4a6bfca60d54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=370d515a06ec468ea437e3b31a556dcb)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:16.330774 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299439a-4ca0-4b61-b2bd-4a6bfca60d54 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:16.336619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-8b07b913-a484-4466-bac3-bab9eb422ef8 None None] MFA Rules not processed for user `7155dd4b7f684400a85f02709e774d7f`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:16.338140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-8b07b913-a484-4466-bac3-bab9eb422ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:16.417263 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2299439a-4ca0-4b61-b2bd-4a6bfca60d54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '370d515a06ec468ea437e3b31a556dcb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:16.426397 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2299439a-4ca0-4b61-b2bd-4a6bfca60d54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '370d515a06ec468ea437e3b31a556dcb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:16.427124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2299439a-4ca0-4b61-b2bd-4a6bfca60d54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '370d515a06ec468ea437e3b31a556dcb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:16.439762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2299439a-4ca0-4b61-b2bd-4a6bfca60d54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '370d515a06ec468ea437e3b31a556dcb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:16.440577 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2299439a-4ca0-4b61-b2bd-4a6bfca60d54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '370d515a06ec468ea437e3b31a556dcb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:16.481909 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 924/1887] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:16 2022] DELETE /identity/v3/users/370d515a06ec468ea437e3b31a556dcb => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:16.552376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 964/1888] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:16.559572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299439a-4ca0-4b61-b2bd-4a6bfca60d54 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:16.562246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-2299439a-4ca0-4b61-b2bd-4a6bfca60d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:16.712239 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299439a-4ca0-4b61-b2bd-4a6bfca60d54 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:16.718733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783f22c-1ea5-46c5-9a27-58161ea51dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:16.719809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783f22c-1ea5-46c5-9a27-58161ea51dad admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:16.720169 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783f22c-1ea5-46c5-9a27-58161ea51dad admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:16.720564 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783f22c-1ea5-46c5-9a27-58161ea51dad admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:16.721339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783f22c-1ea5-46c5-9a27-58161ea51dad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:16.723345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783f22c-1ea5-46c5-9a27-58161ea51dad admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:16.790409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 925/1889] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:16 2022] POST /identity/v3/projects => generated 401 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:16.793802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07b913-a484-4466-bac3-bab9eb422ef8 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:16.809967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07b913-a484-4466-bac3-bab9eb422ef8 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:16.915373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b794f62-de49-4b21-962f-41e0f241d87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:16.916275 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b794f62-de49-4b21-962f-41e0f241d87a admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:16.916661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b794f62-de49-4b21-962f-41e0f241d87a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:16.917068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b794f62-de49-4b21-962f-41e0f241d87a admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:16.918079 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b794f62-de49-4b21-962f-41e0f241d87a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:16.920638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b794f62-de49-4b21-962f-41e0f241d87a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:16.946766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 965/1890] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:16 2022] POST /identity/v3/users => generated 307 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:16.950207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783f22c-1ea5-46c5-9a27-58161ea51dad admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:16.962265 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783f22c-1ea5-46c5-9a27-58161ea51dad admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:16.973010 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0654450-b04c-4176-8c62-115f2355f501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:16.973998 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0654450-b04c-4176-8c62-115f2355f501 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:16.974718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0654450-b04c-4176-8c62-115f2355f501 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:16.975214 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0654450-b04c-4176-8c62-115f2355f501 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:16.976287 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0654450-b04c-4176-8c62-115f2355f501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:16.978200 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0654450-b04c-4176-8c62-115f2355f501 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:17.006299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b794f62-de49-4b21-962f-41e0f241d87a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:17.015168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 926/1891] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:17.020219 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b794f62-de49-4b21-962f-41e0f241d87a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:17.028638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f553483-7bdd-4522-aa8e-ee3b74959427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:17.030640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f553483-7bdd-4522-aa8e-ee3b74959427 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:17.031540 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f553483-7bdd-4522-aa8e-ee3b74959427 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:17.032673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f553483-7bdd-4522-aa8e-ee3b74959427 admin admin] PATH_INFO: `/v3/projects/d617c60d798944fea1677bb7c070964a/users/e856c2a20bc94f46aa9cf12243a17be1/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:17.055491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f553483-7bdd-4522-aa8e-ee3b74959427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d617c60d798944fea1677bb7c070964a, user_id=e856c2a20bc94f46aa9cf12243a17be1, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:17.059802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f553483-7bdd-4522-aa8e-ee3b74959427 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:17.086512 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 966/1892] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:17 2022] PUT /identity/v3/projects/d617c60d798944fea1677bb7c070964a/users/e856c2a20bc94f46aa9cf12243a17be1/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:17.091492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0654450-b04c-4176-8c62-115f2355f501 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:17.107757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0654450-b04c-4176-8c62-115f2355f501 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:17.240567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e30ca5-301e-42b5-922e-458adef1d442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:17.242169 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30ca5-301e-42b5-922e-458adef1d442 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:17.246396 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30ca5-301e-42b5-922e-458adef1d442 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:17.247300 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30ca5-301e-42b5-922e-458adef1d442 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:17.250349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e30ca5-301e-42b5-922e-458adef1d442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:17.253363 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e30ca5-301e-42b5-922e-458adef1d442 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:17.282307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 927/1893] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:17.284465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f553483-7bdd-4522-aa8e-ee3b74959427 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:17.300774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f553483-7bdd-4522-aa8e-ee3b74959427 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:17.311094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b590c97-5adc-461d-ba57-d384f5b2ed71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:17.313183 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b590c97-5adc-461d-ba57-d384f5b2ed71 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:17.313673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b590c97-5adc-461d-ba57-d384f5b2ed71 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:17.314155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b590c97-5adc-461d-ba57-d384f5b2ed71 admin admin] PATH_INFO: `/v3/projects/d617c60d798944fea1677bb7c070964a/users/e856c2a20bc94f46aa9cf12243a17be1/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:17.319676 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b590c97-5adc-461d-ba57-d384f5b2ed71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d617c60d798944fea1677bb7c070964a, user_id=e856c2a20bc94f46aa9cf12243a17be1, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:17.323086 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b590c97-5adc-461d-ba57-d384f5b2ed71 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:17.344386 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 967/1894] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:17 2022] PUT /identity/v3/projects/d617c60d798944fea1677bb7c070964a/users/e856c2a20bc94f46aa9cf12243a17be1/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:17.621534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e30ca5-301e-42b5-922e-458adef1d442 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:17.625500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c1e30ca5-301e-42b5-922e-458adef1d442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:17.648074 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e30ca5-301e-42b5-922e-458adef1d442 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:17.752729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a8e90-ea36-4bb7-b401-587a19329bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:17.753678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a8e90-ea36-4bb7-b401-587a19329bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:17.754099 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a8e90-ea36-4bb7-b401-587a19329bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:17.754446 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a8e90-ea36-4bb7-b401-587a19329bf1 admin admin] PATH_INFO: `/v3/projects/aa5f55a14cd8438e898de970686597f6` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:17.755766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a8e90-ea36-4bb7-b401-587a19329bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa5f55a14cd8438e898de970686597f6)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:17.763447 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a8e90-ea36-4bb7-b401-587a19329bf1 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:17.839986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b590c97-5adc-461d-ba57-d384f5b2ed71 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:17.842087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2b590c97-5adc-461d-ba57-d384f5b2ed71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:17.845536 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-957a8e90-ea36-4bb7-b401-587a19329bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa5f55a14cd8438e898de970686597f6', 'request_id': 'req-957a8e90-ea36-4bb7-b401-587a19329bf1'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:17.854557 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-957a8e90-ea36-4bb7-b401-587a19329bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa5f55a14cd8438e898de970686597f6', 'request_id': 'req-957a8e90-ea36-4bb7-b401-587a19329bf1'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:17.867804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-957a8e90-ea36-4bb7-b401-587a19329bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa5f55a14cd8438e898de970686597f6', 'request_id': 'req-957a8e90-ea36-4bb7-b401-587a19329bf1'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:17.870797 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b590c97-5adc-461d-ba57-d384f5b2ed71 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:17.885042 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-957a8e90-ea36-4bb7-b401-587a19329bf1 admin admin] The token cache is being invalidate because project aa5f55a14cd8438e898de970686597f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:17.885742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-957a8e90-ea36-4bb7-b401-587a19329bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:17.887368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-957a8e90-ea36-4bb7-b401-587a19329bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:17.894598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 928/1895] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:17 2022] DELETE /identity/v3/projects/aa5f55a14cd8438e898de970686597f6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:17.897124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af857656-8949-4e88-a04c-637a00052d01 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:17.897601 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af857656-8949-4e88-a04c-637a00052d01 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:17.898056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af857656-8949-4e88-a04c-637a00052d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:17.955163 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-af857656-8949-4e88-a04c-637a00052d01 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:17.956841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-af857656-8949-4e88-a04c-637a00052d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:18.012344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07860a2f-e6c6-420c-9a0f-d43abe2cbb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:18.013413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07860a2f-e6c6-420c-9a0f-d43abe2cbb8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:18.013695 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07860a2f-e6c6-420c-9a0f-d43abe2cbb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:18.014068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07860a2f-e6c6-420c-9a0f-d43abe2cbb8d admin admin] PATH_INFO: `/v3/users/01c5c4a7b0524952a904cf229073a953` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:18.015342 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07860a2f-e6c6-420c-9a0f-d43abe2cbb8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c5c4a7b0524952a904cf229073a953)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:18.017754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07860a2f-e6c6-420c-9a0f-d43abe2cbb8d admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:18.089222 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-07860a2f-e6c6-420c-9a0f-d43abe2cbb8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c5c4a7b0524952a904cf229073a953'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:18.090346 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-07860a2f-e6c6-420c-9a0f-d43abe2cbb8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c5c4a7b0524952a904cf229073a953'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:18.102097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-07860a2f-e6c6-420c-9a0f-d43abe2cbb8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c5c4a7b0524952a904cf229073a953'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:18.110746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-07860a2f-e6c6-420c-9a0f-d43abe2cbb8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c5c4a7b0524952a904cf229073a953'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:18.111641 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-07860a2f-e6c6-420c-9a0f-d43abe2cbb8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c5c4a7b0524952a904cf229073a953'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:18.121129 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 929/1896] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:18.129775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae0896-fb3a-4e6c-84c1-6a9357b95ff7 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:18.130139 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae0896-fb3a-4e6c-84c1-6a9357b95ff7 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:18.130331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae0896-fb3a-4e6c-84c1-6a9357b95ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:18.144314 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 968/1897] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:17 2022] DELETE /identity/v3/users/01c5c4a7b0524952a904cf229073a953 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:18.149075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a1953-6d37-4748-96a1-d6dc1977f84c None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:18.149418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a1953-6d37-4748-96a1-d6dc1977f84c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:18.149700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a1953-6d37-4748-96a1-d6dc1977f84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:18.159331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ffae0896-fb3a-4e6c-84c1-6a9357b95ff7 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:18.160720 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ffae0896-fb3a-4e6c-84c1-6a9357b95ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:18.171164 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-972a1953-6d37-4748-96a1-d6dc1977f84c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:18.172755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-972a1953-6d37-4748-96a1-d6dc1977f84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:18.226581 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 930/1898] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:18.230711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae0896-fb3a-4e6c-84c1-6a9357b95ff7 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:18.242009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae0896-fb3a-4e6c-84c1-6a9357b95ff7 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:18.247023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dfcc2-ed38-45a3-8fae-23a272dcef03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:18.247922 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dfcc2-ed38-45a3-8fae-23a272dcef03 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:18.248206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dfcc2-ed38-45a3-8fae-23a272dcef03 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:18.251837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dfcc2-ed38-45a3-8fae-23a272dcef03 admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:18.251971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 969/1899] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:18.252319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dfcc2-ed38-45a3-8fae-23a272dcef03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:30:18.252319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dfcc2-ed38-45a3-8fae-23a272dcef03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:18.252593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dfcc2-ed38-45a3-8fae-23a272dcef03 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:18.259477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a1953-6d37-4748-96a1-d6dc1977f84c None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:18.260887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-972a1953-6d37-4748-96a1-d6dc1977f84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:18.268841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 931/1900] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:30:18.270188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dfcc2-ed38-45a3-8fae-23a272dcef03 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:18.281017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dfcc2-ed38-45a3-8fae-23a272dcef03 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:18.306505 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef04a76-3a30-4d9c-ba33-3da72d528aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:18.307663 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef04a76-3a30-4d9c-ba33-3da72d528aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:18.307663 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef04a76-3a30-4d9c-ba33-3da72d528aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:18.307831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef04a76-3a30-4d9c-ba33-3da72d528aa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:18.308467 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef04a76-3a30-4d9c-ba33-3da72d528aa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:18.310716 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef04a76-3a30-4d9c-ba33-3da72d528aa5 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:18.355044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a1953-6d37-4748-96a1-d6dc1977f84c None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:18.360280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff679c-7b2c-4815-9659-ef90edd5a4a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:18.361161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff679c-7b2c-4815-9659-ef90edd5a4a0 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:18.361445 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff679c-7b2c-4815-9659-ef90edd5a4a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:18.361762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff679c-7b2c-4815-9659-ef90edd5a4a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:18.362752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 932/1901] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:18 2022] POST /identity/v3/projects => generated 379 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:30:18.363569 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef04a76-3a30-4d9c-ba33-3da72d528aa5 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:18.374320 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff679c-7b2c-4815-9659-ef90edd5a4a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:18.378283 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff679c-7b2c-4815-9659-ef90edd5a4a0 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:18.397251 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 970/1902] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:30:18.439435 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef04a76-3a30-4d9c-ba33-3da72d528aa5 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:18.444748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99deccb8-0ce5-4d75-94de-f75a9e97f97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:18.445429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deccb8-0ce5-4d75-94de-f75a9e97f97d admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:18.445782 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deccb8-0ce5-4d75-94de-f75a9e97f97d admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:18.446071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deccb8-0ce5-4d75-94de-f75a9e97f97d admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:18.446761 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99deccb8-0ce5-4d75-94de-f75a9e97f97d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:18.448948 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99deccb8-0ce5-4d75-94de-f75a9e97f97d admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:18.484032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 933/1903] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:18 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:18.488254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff679c-7b2c-4815-9659-ef90edd5a4a0 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:18.500613 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff679c-7b2c-4815-9659-ef90edd5a4a0 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:18.509853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e75643-5aa9-4314-8a7a-3407985f417e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:18.510917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e75643-5aa9-4314-8a7a-3407985f417e admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:18.511423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e75643-5aa9-4314-8a7a-3407985f417e admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:18.511927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e75643-5aa9-4314-8a7a-3407985f417e admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:18.513619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e75643-5aa9-4314-8a7a-3407985f417e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:18.516231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e75643-5aa9-4314-8a7a-3407985f417e admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:18.533932 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 971/1904] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:18.536940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99deccb8-0ce5-4d75-94de-f75a9e97f97d admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:18.552095 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99deccb8-0ce5-4d75-94de-f75a9e97f97d admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:18.568767 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496770c-e6bb-4bb8-8b47-e7628593163f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:18.571333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496770c-e6bb-4bb8-8b47-e7628593163f admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:18.573029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496770c-e6bb-4bb8-8b47-e7628593163f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:18.573300 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496770c-e6bb-4bb8-8b47-e7628593163f admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:18.574206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496770c-e6bb-4bb8-8b47-e7628593163f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:18.579120 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496770c-e6bb-4bb8-8b47-e7628593163f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:18.609391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e75643-5aa9-4314-8a7a-3407985f417e admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:18.620174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 934/1905] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:18.621227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e75643-5aa9-4314-8a7a-3407985f417e admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:18.634804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70542c-72dd-4c4f-97b3-10b7996c1467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:18.635785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70542c-72dd-4c4f-97b3-10b7996c1467 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:18.636069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70542c-72dd-4c4f-97b3-10b7996c1467 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:18.636341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70542c-72dd-4c4f-97b3-10b7996c1467 admin admin] PATH_INFO: `/v3/projects/071ef0d388954d70ad3129c58d81d756/users/2f99157ab7f94780a11fb74441adf632/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:18.673380 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70542c-72dd-4c4f-97b3-10b7996c1467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=071ef0d388954d70ad3129c58d81d756, user_id=2f99157ab7f94780a11fb74441adf632, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:18.678117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70542c-72dd-4c4f-97b3-10b7996c1467 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:18.719793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 972/1906] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:18 2022] PUT /identity/v3/projects/071ef0d388954d70ad3129c58d81d756/users/2f99157ab7f94780a11fb74441adf632/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:18.730119 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011a7ba-f06b-4369-b28f-6655453f59ea None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:18.730705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011a7ba-f06b-4369-b28f-6655453f59ea None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:18.730796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011a7ba-f06b-4369-b28f-6655453f59ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:18.777095 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-1011a7ba-f06b-4369-b28f-6655453f59ea None None] MFA Rules not processed for user `2f99157ab7f94780a11fb74441adf632`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:18.778016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1011a7ba-f06b-4369-b28f-6655453f59ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:18.887986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 935/1907] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:18.893037 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70542c-72dd-4c4f-97b3-10b7996c1467 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:19.074958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70542c-72dd-4c4f-97b3-10b7996c1467 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:19.085427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5a506-578e-44a3-9ce8-683bd10bb694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:19.086116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5a506-578e-44a3-9ce8-683bd10bb694 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:19.086398 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5a506-578e-44a3-9ce8-683bd10bb694 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:19.086674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5a506-578e-44a3-9ce8-683bd10bb694 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:19.088117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5a506-578e-44a3-9ce8-683bd10bb694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:19.089812 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5a506-578e-44a3-9ce8-683bd10bb694 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:19.197883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 973/1908] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:18 2022] POST /identity/v3/projects => generated 379 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:19.203133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011a7ba-f06b-4369-b28f-6655453f59ea None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:19.229434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011a7ba-f06b-4369-b28f-6655453f59ea None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:19.239513 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5a506-578e-44a3-9ce8-683bd10bb694 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:19.248259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-fcb5a506-578e-44a3-9ce8-683bd10bb694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:19.364427 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723897b9-4c41-4a33-af91-03c7211efa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:19.365087 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723897b9-4c41-4a33-af91-03c7211efa29 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:19.365373 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723897b9-4c41-4a33-af91-03c7211efa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:19.365643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723897b9-4c41-4a33-af91-03c7211efa29 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:19.366676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723897b9-4c41-4a33-af91-03c7211efa29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:19.370071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723897b9-4c41-4a33-af91-03c7211efa29 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:19.372208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5a506-578e-44a3-9ce8-683bd10bb694 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:19.378128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:19.379007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:19.379381 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:19.379652 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] PATH_INFO: `/v3/projects/93ecde25b0754b23a8a40ee9ab9219c3` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:19.380907 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ecde25b0754b23a8a40ee9ab9219c3)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:19.383021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:19.417670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 936/1909] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:19 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:30:19.420025 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46530a-8663-484f-baba-c44ca498a014 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:19.420481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46530a-8663-484f-baba-c44ca498a014 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:19.420884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46530a-8663-484f-baba-c44ca498a014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:19.450869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ecde25b0754b23a8a40ee9ab9219c3', 'request_id': 'req-356fa7c8-092a-41f6-a3dc-ed498b1e641a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:19.452562 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ed46530a-8663-484f-baba-c44ca498a014 None None] MFA Rules not processed for user `e856c2a20bc94f46aa9cf12243a17be1`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:19.460900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ed46530a-8663-484f-baba-c44ca498a014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:19.461133 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ecde25b0754b23a8a40ee9ab9219c3', 'request_id': 'req-356fa7c8-092a-41f6-a3dc-ed498b1e641a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:19.467438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ecde25b0754b23a8a40ee9ab9219c3', 'request_id': 'req-356fa7c8-092a-41f6-a3dc-ed498b1e641a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:19.480205 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] The token cache is being invalidate because project 93ecde25b0754b23a8a40ee9ab9219c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:19.480629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:19.481553 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:19.483923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 974/1910] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:19 2022] DELETE /identity/v3/projects/93ecde25b0754b23a8a40ee9ab9219c3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:19.487402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:19.488921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:19.545483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 937/1911] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:19 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:19.545483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46530a-8663-484f-baba-c44ca498a014 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:19.545483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ed46530a-8663-484f-baba-c44ca498a014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:19.569148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fa7c8-092a-41f6-a3dc-ed498b1e641a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:19.581936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21aaaa-a36b-4969-977f-ee914e3973ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:19.583563 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21aaaa-a36b-4969-977f-ee914e3973ce admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:19.584214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21aaaa-a36b-4969-977f-ee914e3973ce admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:19.584782 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21aaaa-a36b-4969-977f-ee914e3973ce admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:19.586448 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21aaaa-a36b-4969-977f-ee914e3973ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:19.588796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21aaaa-a36b-4969-977f-ee914e3973ce admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:19.605864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 975/1912] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:19.607033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21aaaa-a36b-4969-977f-ee914e3973ce admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:19.619817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21aaaa-a36b-4969-977f-ee914e3973ce admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:19.627471 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f58ad-8b8d-46a0-8e25-0ab1523ebe43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:19.627931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f58ad-8b8d-46a0-8e25-0ab1523ebe43 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:19.628200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f58ad-8b8d-46a0-8e25-0ab1523ebe43 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:19.628475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f58ad-8b8d-46a0-8e25-0ab1523ebe43 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:19.629550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f58ad-8b8d-46a0-8e25-0ab1523ebe43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:19.631209 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f58ad-8b8d-46a0-8e25-0ab1523ebe43 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:19.637841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46530a-8663-484f-baba-c44ca498a014 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:19.643852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53307f55-36d8-4bb7-a2dd-5cafb099832c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:19.644134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 976/1913] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:19.644889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53307f55-36d8-4bb7-a2dd-5cafb099832c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:19.645248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53307f55-36d8-4bb7-a2dd-5cafb099832c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:19.646148 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53307f55-36d8-4bb7-a2dd-5cafb099832c admin admin] PATH_INFO: `/v3/users/aa687bdff8b14523a839024ecd8eb5a4` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:19.646996 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53307f55-36d8-4bb7-a2dd-5cafb099832c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa687bdff8b14523a839024ecd8eb5a4)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:19.648426 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f58ad-8b8d-46a0-8e25-0ab1523ebe43 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:19.649902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53307f55-36d8-4bb7-a2dd-5cafb099832c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:19.659989 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f58ad-8b8d-46a0-8e25-0ab1523ebe43 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:19.665933 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217d3f5-5a99-4e63-9721-500aa4d30452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:19.666988 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217d3f5-5a99-4e63-9721-500aa4d30452 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:19.667298 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217d3f5-5a99-4e63-9721-500aa4d30452 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:19.667864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217d3f5-5a99-4e63-9721-500aa4d30452 admin admin] PATH_INFO: `/v3/projects/ea22e236a132454591b3a93c707554ab/users/42d1f33168484b35a5c26959a66d2f40/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:19.701207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217d3f5-5a99-4e63-9721-500aa4d30452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea22e236a132454591b3a93c707554ab, user_id=42d1f33168484b35a5c26959a66d2f40, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:19.703944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217d3f5-5a99-4e63-9721-500aa4d30452 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:19.727123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-53307f55-36d8-4bb7-a2dd-5cafb099832c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa687bdff8b14523a839024ecd8eb5a4'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:19.729241 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 977/1914] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:19 2022] PUT /identity/v3/projects/ea22e236a132454591b3a93c707554ab/users/42d1f33168484b35a5c26959a66d2f40/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:19.741691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a0ed6-65a1-41ea-8c0d-1d263c187d6c None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:19.742324 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a0ed6-65a1-41ea-8c0d-1d263c187d6c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:19.743096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a0ed6-65a1-41ea-8c0d-1d263c187d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:19.745184 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-53307f55-36d8-4bb7-a2dd-5cafb099832c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa687bdff8b14523a839024ecd8eb5a4'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:19.746135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-53307f55-36d8-4bb7-a2dd-5cafb099832c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa687bdff8b14523a839024ecd8eb5a4'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:19.760954 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-53307f55-36d8-4bb7-a2dd-5cafb099832c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa687bdff8b14523a839024ecd8eb5a4'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:19.762273 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-53307f55-36d8-4bb7-a2dd-5cafb099832c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa687bdff8b14523a839024ecd8eb5a4'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:19.773236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-dd2a0ed6-65a1-41ea-8c0d-1d263c187d6c None None] MFA Rules not processed for user `42d1f33168484b35a5c26959a66d2f40`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:19.774538 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-dd2a0ed6-65a1-41ea-8c0d-1d263c187d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:19.800504 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 938/1915] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:19 2022] DELETE /identity/v3/users/aa687bdff8b14523a839024ecd8eb5a4 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:19.865484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 978/1916] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:19.868932 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53307f55-36d8-4bb7-a2dd-5cafb099832c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:19.870365 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-53307f55-36d8-4bb7-a2dd-5cafb099832c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:19.972965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53307f55-36d8-4bb7-a2dd-5cafb099832c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:19.978705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95abf4-74c1-4586-8954-5e74002c2bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:19.979493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95abf4-74c1-4586-8954-5e74002c2bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:19.979843 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95abf4-74c1-4586-8954-5e74002c2bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:19.980189 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95abf4-74c1-4586-8954-5e74002c2bc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:19.980964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d95abf4-74c1-4586-8954-5e74002c2bc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:19.982978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d95abf4-74c1-4586-8954-5e74002c2bc3 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:20.016275 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 939/1917] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:19 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:20.020521 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a0ed6-65a1-41ea-8c0d-1d263c187d6c None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:20.095598 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a0ed6-65a1-41ea-8c0d-1d263c187d6c None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:20.221018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debaa4d1-042d-46a5-89c9-1a5da675752a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:20.223706 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debaa4d1-042d-46a5-89c9-1a5da675752a admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:20.224116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debaa4d1-042d-46a5-89c9-1a5da675752a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:20.224496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debaa4d1-042d-46a5-89c9-1a5da675752a admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:20.225292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debaa4d1-042d-46a5-89c9-1a5da675752a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:20.227423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debaa4d1-042d-46a5-89c9-1a5da675752a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:20.249039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95abf4-74c1-4586-8954-5e74002c2bc3 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:20.251593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7d95abf4-74c1-4586-8954-5e74002c2bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:20.257766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 979/1918] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:20 2022] POST /identity/v3/users => generated 296 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:30:20.261643 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debaa4d1-042d-46a5-89c9-1a5da675752a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:20.280690 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95abf4-74c1-4586-8954-5e74002c2bc3 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:20.283520 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debaa4d1-042d-46a5-89c9-1a5da675752a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:20.291691 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:20.292586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:20.292872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:20.293152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] PATH_INFO: `/v3/projects/3e3c8e07e00841d385759c2012714bd8` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:20.293725 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e905892-9621-4643-a945-856e3a67bcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:20.294688 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e905892-9621-4643-a945-856e3a67bcc3 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:20.295033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e905892-9621-4643-a945-856e3a67bcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:20.295442 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e905892-9621-4643-a945-856e3a67bcc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:20.296227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e905892-9621-4643-a945-856e3a67bcc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:20.298825 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e905892-9621-4643-a945-856e3a67bcc3 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:20.303573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e3c8e07e00841d385759c2012714bd8)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:20.308159 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:20.312611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 980/1919] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:30:20.315931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e905892-9621-4643-a945-856e3a67bcc3 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:20.329874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e905892-9621-4643-a945-856e3a67bcc3 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:20.336497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ec52e-e4cc-4267-a0a3-5b00975070d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:20.337520 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ec52e-e4cc-4267-a0a3-5b00975070d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:20.337806 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ec52e-e4cc-4267-a0a3-5b00975070d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:20.338079 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ec52e-e4cc-4267-a0a3-5b00975070d8 admin admin] PATH_INFO: `/v3/projects/6895f7368f3a4472b18b7d2d27943209/users/b0d1720b2a2d4c269ae68c0c01dc70ac/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:20.358628 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ec52e-e4cc-4267-a0a3-5b00975070d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6895f7368f3a4472b18b7d2d27943209, user_id=b0d1720b2a2d4c269ae68c0c01dc70ac, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:20.362523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ec52e-e4cc-4267-a0a3-5b00975070d8 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:20.391024 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 981/1920] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:20 2022] PUT /identity/v3/projects/6895f7368f3a4472b18b7d2d27943209/users/b0d1720b2a2d4c269ae68c0c01dc70ac/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:20.396005 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ec52e-e4cc-4267-a0a3-5b00975070d8 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:20.420690 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e3c8e07e00841d385759c2012714bd8', 'request_id': 'req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:20.427256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e3c8e07e00841d385759c2012714bd8', 'request_id': 'req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:20.439055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e3c8e07e00841d385759c2012714bd8', 'request_id': 'req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:20.454693 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] The token cache is being invalidate because project 3e3c8e07e00841d385759c2012714bd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:20.455032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:20.455745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:20.457816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 940/1921] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:20 2022] DELETE /identity/v3/projects/3e3c8e07e00841d385759c2012714bd8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:20.463128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:20.464752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:20.488801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ec52e-e4cc-4267-a0a3-5b00975070d8 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:20.494431 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a10396-5bf3-4ec0-843e-27b96990bbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:20.495205 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10396-5bf3-4ec0-843e-27b96990bbb9 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:20.495566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10396-5bf3-4ec0-843e-27b96990bbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:20.495969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10396-5bf3-4ec0-843e-27b96990bbb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:20.496771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a10396-5bf3-4ec0-843e-27b96990bbb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:20.499378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a10396-5bf3-4ec0-843e-27b96990bbb9 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:20.514819 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 982/1922] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:20.520137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a10396-5bf3-4ec0-843e-27b96990bbb9 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:20.526011 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-07a10396-5bf3-4ec0-843e-27b96990bbb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:20.547893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a10396-5bf3-4ec0-843e-27b96990bbb9 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:20.553612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f930fb3-7a42-45ec-a406-7e94f9a9a7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:20.554586 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f930fb3-7a42-45ec-a406-7e94f9a9a7a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:20.554969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f930fb3-7a42-45ec-a406-7e94f9a9a7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:20.555509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f930fb3-7a42-45ec-a406-7e94f9a9a7a6 admin admin] PATH_INFO: `/v3/projects/6895f7368f3a4472b18b7d2d27943209/users/b0d1720b2a2d4c269ae68c0c01dc70ac/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:20.557816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f930fb3-7a42-45ec-a406-7e94f9a9a7a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6895f7368f3a4472b18b7d2d27943209, user_id=b0d1720b2a2d4c269ae68c0c01dc70ac, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:20.561230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f930fb3-7a42-45ec-a406-7e94f9a9a7a6 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:20.562132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e3b77-e6ad-40e8-bc7c-4307e9210e72 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:20.567416 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a71daa-8b27-49b6-afa4-61858e0ca7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:20.568552 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a71daa-8b27-49b6-afa4-61858e0ca7d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:20.568886 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a71daa-8b27-49b6-afa4-61858e0ca7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:20.569209 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a71daa-8b27-49b6-afa4-61858e0ca7d6 admin admin] PATH_INFO: `/v3/users/493b563a11f74716a54b826c481ac7d2` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:20.570432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a71daa-8b27-49b6-afa4-61858e0ca7d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=493b563a11f74716a54b826c481ac7d2)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:20.573396 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a71daa-8b27-49b6-afa4-61858e0ca7d6 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:20.576536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 983/1923] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:20 2022] PUT /identity/v3/projects/6895f7368f3a4472b18b7d2d27943209/users/b0d1720b2a2d4c269ae68c0c01dc70ac/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:20.588257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459a04-caeb-45b6-bc03-ad356014c754 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:20.588831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459a04-caeb-45b6-bc03-ad356014c754 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:20.589444 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459a04-caeb-45b6-bc03-ad356014c754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:20.665639 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b4a71daa-8b27-49b6-afa4-61858e0ca7d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '493b563a11f74716a54b826c481ac7d2'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:20.667699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-dc459a04-caeb-45b6-bc03-ad356014c754 None None] MFA Rules not processed for user `b0d1720b2a2d4c269ae68c0c01dc70ac`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:20.668936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-dc459a04-caeb-45b6-bc03-ad356014c754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:20.686888 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b4a71daa-8b27-49b6-afa4-61858e0ca7d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '493b563a11f74716a54b826c481ac7d2'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:20.691728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b4a71daa-8b27-49b6-afa4-61858e0ca7d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '493b563a11f74716a54b826c481ac7d2'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:20.702797 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b4a71daa-8b27-49b6-afa4-61858e0ca7d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '493b563a11f74716a54b826c481ac7d2'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:20.703477 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b4a71daa-8b27-49b6-afa4-61858e0ca7d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '493b563a11f74716a54b826c481ac7d2'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:20.740079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 941/1924] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:20 2022] DELETE /identity/v3/users/493b563a11f74716a54b826c481ac7d2 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:20.823863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 984/1925] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:20.931961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a71daa-8b27-49b6-afa4-61858e0ca7d6 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:20.932950 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b4a71daa-8b27-49b6-afa4-61858e0ca7d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:21.036256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a71daa-8b27-49b6-afa4-61858e0ca7d6 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:21.040121 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ed38b-ff37-4f3c-a0ae-64547f2d3821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:21.040777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ed38b-ff37-4f3c-a0ae-64547f2d3821 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:21.041059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ed38b-ff37-4f3c-a0ae-64547f2d3821 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:21.041332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ed38b-ff37-4f3c-a0ae-64547f2d3821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:21.109598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ed38b-ff37-4f3c-a0ae-64547f2d3821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:21.111139 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ed38b-ff37-4f3c-a0ae-64547f2d3821 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:21.122706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 942/1926] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:21.175436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459a04-caeb-45b6-bc03-ad356014c754 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:21.176950 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-dc459a04-caeb-45b6-bc03-ad356014c754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:21.254321 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459a04-caeb-45b6-bc03-ad356014c754 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:21.257934 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2980498-06e9-4b35-a409-83e183bfa025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:21.258583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2980498-06e9-4b35-a409-83e183bfa025 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:21.258859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2980498-06e9-4b35-a409-83e183bfa025 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:21.259129 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2980498-06e9-4b35-a409-83e183bfa025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:21.261113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c2980498-06e9-4b35-a409-83e183bfa025 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:21.299359 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ed38b-ff37-4f3c-a0ae-64547f2d3821 service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:21.302865 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ac9ed38b-ff37-4f3c-a0ae-64547f2d3821 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:21.363630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2980498-06e9-4b35-a409-83e183bfa025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:21.365339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2980498-06e9-4b35-a409-83e183bfa025 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:21.386519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 985/1927] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:21.394669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ed38b-ff37-4f3c-a0ae-64547f2d3821 service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:21.401917 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa899c99-cce8-4376-9061-116f35b98a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:21.404751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa899c99-cce8-4376-9061-116f35b98a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:21.405330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa899c99-cce8-4376-9061-116f35b98a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:21.405594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa899c99-cce8-4376-9061-116f35b98a9a admin admin] PATH_INFO: `/v3/projects/1ccf0c7cabe44eebb95f4022a8ec52f9` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:21.407048 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa899c99-cce8-4376-9061-116f35b98a9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ccf0c7cabe44eebb95f4022a8ec52f9)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:21.410976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa899c99-cce8-4376-9061-116f35b98a9a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:21.455137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2980498-06e9-4b35-a409-83e183bfa025 service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:21.458267 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c2980498-06e9-4b35-a409-83e183bfa025 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:21.471493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-aa899c99-cce8-4376-9061-116f35b98a9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ccf0c7cabe44eebb95f4022a8ec52f9', 'request_id': 'req-aa899c99-cce8-4376-9061-116f35b98a9a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:21.477845 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2980498-06e9-4b35-a409-83e183bfa025 service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:21.480104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-aa899c99-cce8-4376-9061-116f35b98a9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ccf0c7cabe44eebb95f4022a8ec52f9', 'request_id': 'req-aa899c99-cce8-4376-9061-116f35b98a9a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:21.488031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-aa899c99-cce8-4376-9061-116f35b98a9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ccf0c7cabe44eebb95f4022a8ec52f9', 'request_id': 'req-aa899c99-cce8-4376-9061-116f35b98a9a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:21.500646 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-aa899c99-cce8-4376-9061-116f35b98a9a admin admin] The token cache is being invalidate because project 1ccf0c7cabe44eebb95f4022a8ec52f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:21.501015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-aa899c99-cce8-4376-9061-116f35b98a9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:21.501722 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-aa899c99-cce8-4376-9061-116f35b98a9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:21.506053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 943/1928] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:21 2022] DELETE /identity/v3/projects/1ccf0c7cabe44eebb95f4022a8ec52f9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:21.511489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0febe0-0b72-46f7-b555-f3066762a5c8 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:21.512415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0febe0-0b72-46f7-b555-f3066762a5c8 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:21.515657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0febe0-0b72-46f7-b555-f3066762a5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:21.538306 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-dd0febe0-0b72-46f7-b555-f3066762a5c8 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:21.540136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-dd0febe0-0b72-46f7-b555-f3066762a5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:21.550016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65c09b-424a-4fe6-ab87-8c533c663cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:21.551070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65c09b-424a-4fe6-ab87-8c533c663cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:21.551377 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65c09b-424a-4fe6-ab87-8c533c663cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:21.551641 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65c09b-424a-4fe6-ab87-8c533c663cf3 admin admin] PATH_INFO: `/v3/users/7155dd4b7f684400a85f02709e774d7f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:21.552822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65c09b-424a-4fe6-ab87-8c533c663cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7155dd4b7f684400a85f02709e774d7f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:21.554714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65c09b-424a-4fe6-ab87-8c533c663cf3 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:21.606178 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-4b65c09b-424a-4fe6-ab87-8c533c663cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7155dd4b7f684400a85f02709e774d7f'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:21.606911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-4b65c09b-424a-4fe6-ab87-8c533c663cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7155dd4b7f684400a85f02709e774d7f'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:21.609771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 944/1929] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:21.614316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-4b65c09b-424a-4fe6-ab87-8c533c663cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7155dd4b7f684400a85f02709e774d7f'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:21.620435 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423ca75-4a7a-4d3e-8299-0ab2f4f1e68d None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:21.620733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423ca75-4a7a-4d3e-8299-0ab2f4f1e68d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:21.621102 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423ca75-4a7a-4d3e-8299-0ab2f4f1e68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:21.622701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-4b65c09b-424a-4fe6-ab87-8c533c663cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7155dd4b7f684400a85f02709e774d7f'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:21.624238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-4b65c09b-424a-4fe6-ab87-8c533c663cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7155dd4b7f684400a85f02709e774d7f'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:21.644791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-c423ca75-4a7a-4d3e-8299-0ab2f4f1e68d None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:21.646107 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c423ca75-4a7a-4d3e-8299-0ab2f4f1e68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:21.647246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 986/1930] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:21 2022] DELETE /identity/v3/users/7155dd4b7f684400a85f02709e774d7f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:21.774828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 945/1931] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:21.779287 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65c09b-424a-4fe6-ab87-8c533c663cf3 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:21.790564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65c09b-424a-4fe6-ab87-8c533c663cf3 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:21.803555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc5aaf-b8b4-4c3f-8219-1f39a31e05af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:21.804535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc5aaf-b8b4-4c3f-8219-1f39a31e05af admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:21.804826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc5aaf-b8b4-4c3f-8219-1f39a31e05af admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:21.805126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc5aaf-b8b4-4c3f-8219-1f39a31e05af admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:21.806734 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc5aaf-b8b4-4c3f-8219-1f39a31e05af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:30:21.807056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc5aaf-b8b4-4c3f-8219-1f39a31e05af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:21.809240 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc5aaf-b8b4-4c3f-8219-1f39a31e05af admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:21.836370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 987/1932] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:30:21.842411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423ca75-4a7a-4d3e-8299-0ab2f4f1e68d None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:21.852564 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423ca75-4a7a-4d3e-8299-0ab2f4f1e68d None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:21.859876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da122a1-6a43-4df5-856a-f088c5b44186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:21.860705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da122a1-6a43-4df5-856a-f088c5b44186 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:21.861195 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da122a1-6a43-4df5-856a-f088c5b44186 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:21.861684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da122a1-6a43-4df5-856a-f088c5b44186 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:21.863128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da122a1-6a43-4df5-856a-f088c5b44186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:21.864673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da122a1-6a43-4df5-856a-f088c5b44186 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:21.895249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 946/1933] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:21 2022] POST /identity/v3/projects => generated 401 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:21.897425 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc5aaf-b8b4-4c3f-8219-1f39a31e05af admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:21.907116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc5aaf-b8b4-4c3f-8219-1f39a31e05af admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:21.976947 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f14d4-13d6-4fa0-b5bf-ab29dd1f9a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:21.977624 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f14d4-13d6-4fa0-b5bf-ab29dd1f9a44 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:21.977903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f14d4-13d6-4fa0-b5bf-ab29dd1f9a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:21.978169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f14d4-13d6-4fa0-b5bf-ab29dd1f9a44 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:21.979032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f14d4-13d6-4fa0-b5bf-ab29dd1f9a44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:21.980761 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f14d4-13d6-4fa0-b5bf-ab29dd1f9a44 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:22.006694 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 988/1934] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:21 2022] POST /identity/v3/users => generated 301 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:22.007852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da122a1-6a43-4df5-856a-f088c5b44186 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:22.020961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da122a1-6a43-4df5-856a-f088c5b44186 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:22.026353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d931fd6-6e24-48c7-8580-bed0690a926a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:22.027036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d931fd6-6e24-48c7-8580-bed0690a926a admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:22.027359 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d931fd6-6e24-48c7-8580-bed0690a926a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:22.027639 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d931fd6-6e24-48c7-8580-bed0690a926a admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:22.028363 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d931fd6-6e24-48c7-8580-bed0690a926a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:22.030048 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d931fd6-6e24-48c7-8580-bed0690a926a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:22.043214 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 947/1935] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:22.046461 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f14d4-13d6-4fa0-b5bf-ab29dd1f9a44 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:22.066595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f14d4-13d6-4fa0-b5bf-ab29dd1f9a44 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:22.066595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a67b8a-4a20-40a2-97d0-aa956d1cce60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:22.066595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a67b8a-4a20-40a2-97d0-aa956d1cce60 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:22.066595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a67b8a-4a20-40a2-97d0-aa956d1cce60 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:22.067055 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a67b8a-4a20-40a2-97d0-aa956d1cce60 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:22.067055 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a67b8a-4a20-40a2-97d0-aa956d1cce60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:22.077971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a67b8a-4a20-40a2-97d0-aa956d1cce60 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:22.099288 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 989/1936] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:22.103773 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d931fd6-6e24-48c7-8580-bed0690a926a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:22.114558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d931fd6-6e24-48c7-8580-bed0690a926a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:22.122227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dde01-7ade-43b8-a780-9966db63dba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:22.122227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dde01-7ade-43b8-a780-9966db63dba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:22.122227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dde01-7ade-43b8-a780-9966db63dba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:22.122227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dde01-7ade-43b8-a780-9966db63dba1 admin admin] PATH_INFO: `/v3/projects/4c6dbd06737d4b8b84a3d09e929e9eb8/users/c322a8ad6388470c99b0248e938d8316/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:22.124322 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a67b8a-4a20-40a2-97d0-aa956d1cce60 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:22.126436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-d4a67b8a-4a20-40a2-97d0-aa956d1cce60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:22.142800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a67b8a-4a20-40a2-97d0-aa956d1cce60 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:22.149169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:22.150044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:22.150349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:22.150616 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] PATH_INFO: `/v3/projects/41f92d3057864904aa809c178c99789a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:22.152175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f92d3057864904aa809c178c99789a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:22.154298 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:22.156467 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dde01-7ade-43b8-a780-9966db63dba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6dbd06737d4b8b84a3d09e929e9eb8, user_id=c322a8ad6388470c99b0248e938d8316, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:22.159368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dde01-7ade-43b8-a780-9966db63dba1 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:22.175626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 948/1937] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:22 2022] PUT /identity/v3/projects/4c6dbd06737d4b8b84a3d09e929e9eb8/users/c322a8ad6388470c99b0248e938d8316/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:22.187484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cf286-b811-49c8-82ec-9d0e76d59186 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:22.187773 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cf286-b811-49c8-82ec-9d0e76d59186 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:22.188053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cf286-b811-49c8-82ec-9d0e76d59186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:22.214031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f92d3057864904aa809c178c99789a', 'request_id': 'req-3b8814b3-3889-43ea-8199-9e98c568bd81'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.217384 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-a11cf286-b811-49c8-82ec-9d0e76d59186 None None] MFA Rules not processed for user `c322a8ad6388470c99b0248e938d8316`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:22.218738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a11cf286-b811-49c8-82ec-9d0e76d59186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:22.219408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f92d3057864904aa809c178c99789a', 'request_id': 'req-3b8814b3-3889-43ea-8199-9e98c568bd81'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.227567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f92d3057864904aa809c178c99789a', 'request_id': 'req-3b8814b3-3889-43ea-8199-9e98c568bd81'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.239741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] The token cache is being invalidate because project 41f92d3057864904aa809c178c99789a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:22.240020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.240697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.242284 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 990/1938] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:22 2022] DELETE /identity/v3/projects/41f92d3057864904aa809c178c99789a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:22.246320 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:22.248787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:22.295325 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 949/1939] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:22.299508 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11cf286-b811-49c8-82ec-9d0e76d59186 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:22.300894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a11cf286-b811-49c8-82ec-9d0e76d59186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:22.321100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8814b3-3889-43ea-8199-9e98c568bd81 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:22.326199 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1b6aa-7f37-44b2-bf71-bb3083cc9acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:22.327771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b6aa-7f37-44b2-bf71-bb3083cc9acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:22.328160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b6aa-7f37-44b2-bf71-bb3083cc9acb admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:22.328443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b6aa-7f37-44b2-bf71-bb3083cc9acb admin admin] PATH_INFO: `/v3/users/e856c2a20bc94f46aa9cf12243a17be1` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:22.329809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1b6aa-7f37-44b2-bf71-bb3083cc9acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e856c2a20bc94f46aa9cf12243a17be1)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:22.331854 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1b6aa-7f37-44b2-bf71-bb3083cc9acb admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:22.387854 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5cb1b6aa-7f37-44b2-bf71-bb3083cc9acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e856c2a20bc94f46aa9cf12243a17be1'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.388480 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5cb1b6aa-7f37-44b2-bf71-bb3083cc9acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e856c2a20bc94f46aa9cf12243a17be1'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.396249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11cf286-b811-49c8-82ec-9d0e76d59186 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:22.400294 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5cb1b6aa-7f37-44b2-bf71-bb3083cc9acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e856c2a20bc94f46aa9cf12243a17be1'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.403956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0694dd2-f0b9-44f6-9eaa-862d5b347c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:22.404708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0694dd2-f0b9-44f6-9eaa-862d5b347c4c admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:22.404998 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0694dd2-f0b9-44f6-9eaa-862d5b347c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:22.405282 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0694dd2-f0b9-44f6-9eaa-862d5b347c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:22.406038 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0694dd2-f0b9-44f6-9eaa-862d5b347c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:22.408116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0694dd2-f0b9-44f6-9eaa-862d5b347c4c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:22.411327 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5cb1b6aa-7f37-44b2-bf71-bb3083cc9acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e856c2a20bc94f46aa9cf12243a17be1'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.412343 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5cb1b6aa-7f37-44b2-bf71-bb3083cc9acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e856c2a20bc94f46aa9cf12243a17be1'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.433180 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 991/1940] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:22 2022] DELETE /identity/v3/users/e856c2a20bc94f46aa9cf12243a17be1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:22.468437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1b6aa-7f37-44b2-bf71-bb3083cc9acb admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:22.469999 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-5cb1b6aa-7f37-44b2-bf71-bb3083cc9acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:22.475396 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 950/1941] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:22 2022] POST /identity/v3/projects => generated 399 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:22.580133 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1b6aa-7f37-44b2-bf71-bb3083cc9acb admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:22.648737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ff8cc-caf6-45db-bfbe-72514fa12d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:22.649348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ff8cc-caf6-45db-bfbe-72514fa12d78 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:22.649734 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ff8cc-caf6-45db-bfbe-72514fa12d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:22.649889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ff8cc-caf6-45db-bfbe-72514fa12d78 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:22.650579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ff8cc-caf6-45db-bfbe-72514fa12d78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:22.653156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ff8cc-caf6-45db-bfbe-72514fa12d78 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:22.690755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 992/1942] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:22 2022] POST /identity/v3/users => generated 306 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:30:22.695742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0694dd2-f0b9-44f6-9eaa-862d5b347c4c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:22.708305 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0694dd2-f0b9-44f6-9eaa-862d5b347c4c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:22.713279 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ae735-3d42-43d7-a67f-e2b29adfb118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:22.713995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ae735-3d42-43d7-a67f-e2b29adfb118 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:22.714275 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ae735-3d42-43d7-a67f-e2b29adfb118 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:22.714549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ae735-3d42-43d7-a67f-e2b29adfb118 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:22.715340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ae735-3d42-43d7-a67f-e2b29adfb118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:22.717475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ae735-3d42-43d7-a67f-e2b29adfb118 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:22.734187 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 951/1943] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:22.735349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ff8cc-caf6-45db-bfbe-72514fa12d78 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:22.744670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ae735-3d42-43d7-a67f-e2b29adfb118 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:22.746747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e80ae735-3d42-43d7-a67f-e2b29adfb118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:22.747609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ff8cc-caf6-45db-bfbe-72514fa12d78 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:22.753293 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a6713-3dc4-4b14-9b11-91ac0a2cda52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:22.754541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a6713-3dc4-4b14-9b11-91ac0a2cda52 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:22.754821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a6713-3dc4-4b14-9b11-91ac0a2cda52 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:22.755098 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a6713-3dc4-4b14-9b11-91ac0a2cda52 admin admin] PATH_INFO: `/v3/projects/6f2992c5de9b451384f54bcd036f05e0/users/1c7e202e69d5412ab61d72654791dcac/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:22.765532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ae735-3d42-43d7-a67f-e2b29adfb118 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:22.770821 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afa86e-348e-4988-8cc0-3178cc625735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:22.771983 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afa86e-348e-4988-8cc0-3178cc625735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:22.772579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afa86e-348e-4988-8cc0-3178cc625735 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:22.773070 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afa86e-348e-4988-8cc0-3178cc625735 admin admin] PATH_INFO: `/v3/projects/d617c60d798944fea1677bb7c070964a` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:22.773197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a6713-3dc4-4b14-9b11-91ac0a2cda52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2992c5de9b451384f54bcd036f05e0, user_id=1c7e202e69d5412ab61d72654791dcac, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:22.774598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afa86e-348e-4988-8cc0-3178cc625735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d617c60d798944fea1677bb7c070964a)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:22.775814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a6713-3dc4-4b14-9b11-91ac0a2cda52 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:22.777214 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afa86e-348e-4988-8cc0-3178cc625735 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:22.793337 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 993/1944] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:22 2022] PUT /identity/v3/projects/6f2992c5de9b451384f54bcd036f05e0/users/1c7e202e69d5412ab61d72654791dcac/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:22.798069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a6713-3dc4-4b14-9b11-91ac0a2cda52 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:22.846147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-43afa86e-348e-4988-8cc0-3178cc625735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd617c60d798944fea1677bb7c070964a', 'request_id': 'req-43afa86e-348e-4988-8cc0-3178cc625735'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.854430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-43afa86e-348e-4988-8cc0-3178cc625735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd617c60d798944fea1677bb7c070964a', 'request_id': 'req-43afa86e-348e-4988-8cc0-3178cc625735'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.865694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-43afa86e-348e-4988-8cc0-3178cc625735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd617c60d798944fea1677bb7c070964a', 'request_id': 'req-43afa86e-348e-4988-8cc0-3178cc625735'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.878650 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-43afa86e-348e-4988-8cc0-3178cc625735 admin admin] The token cache is being invalidate because project d617c60d798944fea1677bb7c070964a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:22.878937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-43afa86e-348e-4988-8cc0-3178cc625735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.879624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-43afa86e-348e-4988-8cc0-3178cc625735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:22.884808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 952/1945] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:22 2022] DELETE /identity/v3/projects/d617c60d798944fea1677bb7c070964a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:22.887227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a6713-3dc4-4b14-9b11-91ac0a2cda52 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:22.890239 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8afff5-a15d-438d-9666-b4d66445c17a None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:22.890547 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8afff5-a15d-438d-9666-b4d66445c17a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:22.890818 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8afff5-a15d-438d-9666-b4d66445c17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:22.910982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-2c8afff5-a15d-438d-9666-b4d66445c17a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:22.912298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-2c8afff5-a15d-438d-9666-b4d66445c17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:22.954635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0a542-4799-470a-8a2e-30e1ec184389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:22.955569 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0a542-4799-470a-8a2e-30e1ec184389 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:22.955859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0a542-4799-470a-8a2e-30e1ec184389 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:22.956218 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0a542-4799-470a-8a2e-30e1ec184389 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:22.957477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0a542-4799-470a-8a2e-30e1ec184389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:22.959068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0a542-4799-470a-8a2e-30e1ec184389 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:22.974791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 994/1946] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:30:22.978833 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0a542-4799-470a-8a2e-30e1ec184389 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:22.980746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-70a0a542-4799-470a-8a2e-30e1ec184389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:22.994038 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0a542-4799-470a-8a2e-30e1ec184389 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:22.999341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea882267-d747-4a9f-988a-35a658f6a50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:22.999478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 953/1947] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:23.000341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea882267-d747-4a9f-988a-35a658f6a50c admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:23.000648 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea882267-d747-4a9f-988a-35a658f6a50c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:23.000960 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea882267-d747-4a9f-988a-35a658f6a50c admin admin] PATH_INFO: `/v3/projects/6f2992c5de9b451384f54bcd036f05e0/users/1c7e202e69d5412ab61d72654791dcac/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:23.008095 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9995b6-6900-4211-b2a7-0e2ab0ec4528 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:23.008592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9995b6-6900-4211-b2a7-0e2ab0ec4528 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:23.008752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea882267-d747-4a9f-988a-35a658f6a50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2992c5de9b451384f54bcd036f05e0, user_id=1c7e202e69d5412ab61d72654791dcac, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:23.008851 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9995b6-6900-4211-b2a7-0e2ab0ec4528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:23.011205 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea882267-d747-4a9f-988a-35a658f6a50c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:23.028565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 995/1948] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:22 2022] PUT /identity/v3/projects/6f2992c5de9b451384f54bcd036f05e0/users/1c7e202e69d5412ab61d72654791dcac/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:23.039202 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-9d9995b6-6900-4211-b2a7-0e2ab0ec4528 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:23.039202 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9d9995b6-6900-4211-b2a7-0e2ab0ec4528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:23.042436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb493c7-6f61-46c0-9b05-4f5587f81fc3 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:23.042797 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb493c7-6f61-46c0-9b05-4f5587f81fc3 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:23.043244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb493c7-6f61-46c0-9b05-4f5587f81fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:23.087287 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2eb493c7-6f61-46c0-9b05-4f5587f81fc3 None None] MFA Rules not processed for user `1c7e202e69d5412ab61d72654791dcac`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:23.089126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2eb493c7-6f61-46c0-9b05-4f5587f81fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:23.161154 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 954/1949] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:23.166500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9995b6-6900-4211-b2a7-0e2ab0ec4528 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:23.186422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9995b6-6900-4211-b2a7-0e2ab0ec4528 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:23.195915 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11986e7-8bad-467b-9fb3-40024e73fffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:23.196900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11986e7-8bad-467b-9fb3-40024e73fffd admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:23.197383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11986e7-8bad-467b-9fb3-40024e73fffd admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:23.197876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11986e7-8bad-467b-9fb3-40024e73fffd admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:23.198809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11986e7-8bad-467b-9fb3-40024e73fffd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:30:23.199966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11986e7-8bad-467b-9fb3-40024e73fffd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:23.202968 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11986e7-8bad-467b-9fb3-40024e73fffd admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:23.225133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 955/1950] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:30:23.229330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11986e7-8bad-467b-9fb3-40024e73fffd admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:23.231341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 996/1951] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:23 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:23.243689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb493c7-6f61-46c0-9b05-4f5587f81fc3 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:23.245677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11986e7-8bad-467b-9fb3-40024e73fffd admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:23.246850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2eb493c7-6f61-46c0-9b05-4f5587f81fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:23.254686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d702caf-ad66-4ff0-ba85-b572af470925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:23.255487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d702caf-ad66-4ff0-ba85-b572af470925 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:23.255841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d702caf-ad66-4ff0-ba85-b572af470925 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:23.256348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d702caf-ad66-4ff0-ba85-b572af470925 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:23.258175 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d702caf-ad66-4ff0-ba85-b572af470925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:23.261048 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d702caf-ad66-4ff0-ba85-b572af470925 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:23.298762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 956/1952] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:23 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:30:23.303596 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d702caf-ad66-4ff0-ba85-b572af470925 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:23.334529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb493c7-6f61-46c0-9b05-4f5587f81fc3 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:23.392221 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d702caf-ad66-4ff0-ba85-b572af470925 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:23.397590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654a548-6134-4464-8a7e-0913d00ac824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:23.398456 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654a548-6134-4464-8a7e-0913d00ac824 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:23.399105 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654a548-6134-4464-8a7e-0913d00ac824 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:23.400708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654a548-6134-4464-8a7e-0913d00ac824 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:23.401434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654a548-6134-4464-8a7e-0913d00ac824 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:23.403389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654a548-6134-4464-8a7e-0913d00ac824 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:23.410245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be34883-074f-4c15-a110-a3283261b189 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:23.410914 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be34883-074f-4c15-a110-a3283261b189 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:23.411562 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be34883-074f-4c15-a110-a3283261b189 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:23.411783 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be34883-074f-4c15-a110-a3283261b189 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:23.453338 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 957/1953] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:23 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:23.454149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654a548-6134-4464-8a7e-0913d00ac824 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:23.467049 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654a548-6134-4464-8a7e-0913d00ac824 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:23.473652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdf1b4-b348-491b-bfa2-752fa2149ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:23.474376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdf1b4-b348-491b-bfa2-752fa2149ceb admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:23.474664 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdf1b4-b348-491b-bfa2-752fa2149ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:23.475346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdf1b4-b348-491b-bfa2-752fa2149ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:23.476128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdf1b4-b348-491b-bfa2-752fa2149ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:23.478416 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdf1b4-b348-491b-bfa2-752fa2149ceb admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:23.493623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 958/1954] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:23.495002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be34883-074f-4c15-a110-a3283261b189 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:23.496893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be34883-074f-4c15-a110-a3283261b189 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:23.497930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdf1b4-b348-491b-bfa2-752fa2149ceb admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:23.507961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdf1b4-b348-491b-bfa2-752fa2149ceb admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:23.508105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 997/1955] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:23 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:23.517449 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143161cd-89e3-44a8-b792-295f369543c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:23.518127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143161cd-89e3-44a8-b792-295f369543c3 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:23.518414 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143161cd-89e3-44a8-b792-295f369543c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:23.518697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143161cd-89e3-44a8-b792-295f369543c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:23.519579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143161cd-89e3-44a8-b792-295f369543c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:23.521296 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143161cd-89e3-44a8-b792-295f369543c3 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:23.533548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 959/1956] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:30:23.537795 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be34883-074f-4c15-a110-a3283261b189 service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:23.554045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be34883-074f-4c15-a110-a3283261b189 service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:23.559461 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e4900-8f4c-4009-962e-874e38edb544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:23.561003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e4900-8f4c-4009-962e-874e38edb544 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:23.561284 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e4900-8f4c-4009-962e-874e38edb544 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:23.561578 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e4900-8f4c-4009-962e-874e38edb544 admin admin] PATH_INFO: `/v3/projects/22e5f97c8b1046c3a66d910502001a8b/users/148bc27efd504ca8997d5b55d61f33ec/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:23.578912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e4900-8f4c-4009-962e-874e38edb544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22e5f97c8b1046c3a66d910502001a8b, user_id=148bc27efd504ca8997d5b55d61f33ec, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:23.581177 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e4900-8f4c-4009-962e-874e38edb544 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:23.597879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143161cd-89e3-44a8-b792-295f369543c3 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:23.601268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 998/1957] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:23 2022] PUT /identity/v3/projects/22e5f97c8b1046c3a66d910502001a8b/users/148bc27efd504ca8997d5b55d61f33ec/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:23.608926 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143161cd-89e3-44a8-b792-295f369543c3 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:23.613238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd30c61-c40a-4d11-a3cc-693cbbb2f8f9 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:23.613533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd30c61-c40a-4d11-a3cc-693cbbb2f8f9 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:23.613808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd30c61-c40a-4d11-a3cc-693cbbb2f8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:23.643259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-3dd30c61-c40a-4d11-a3cc-693cbbb2f8f9 None None] MFA Rules not processed for user `148bc27efd504ca8997d5b55d61f33ec`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:23.644603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3dd30c61-c40a-4d11-a3cc-693cbbb2f8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:23.664780 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f008ca-1a5d-4226-8112-addcfdc2ebab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:23.665953 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f008ca-1a5d-4226-8112-addcfdc2ebab service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:23.666243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f008ca-1a5d-4226-8112-addcfdc2ebab service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:23.666519 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f008ca-1a5d-4226-8112-addcfdc2ebab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:23.674227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f5f008ca-1a5d-4226-8112-addcfdc2ebab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:23.740526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 999/1958] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:23.745100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd30c61-c40a-4d11-a3cc-693cbbb2f8f9 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:23.769596 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f008ca-1a5d-4226-8112-addcfdc2ebab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:23.771099 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f008ca-1a5d-4226-8112-addcfdc2ebab service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:23.781811 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 960/1959] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:23.821771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd30c61-c40a-4d11-a3cc-693cbbb2f8f9 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:23.826971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b3972-6a9d-4d8f-874e-5511c183aa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:23.828385 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b3972-6a9d-4d8f-874e-5511c183aa90 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:23.828385 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b3972-6a9d-4d8f-874e-5511c183aa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:23.828385 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b3972-6a9d-4d8f-874e-5511c183aa90 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:23.829065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b3972-6a9d-4d8f-874e-5511c183aa90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:23.831746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b3972-6a9d-4d8f-874e-5511c183aa90 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:23.861873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1000/1960] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:23 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:23.867478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f008ca-1a5d-4226-8112-addcfdc2ebab service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:23.879716 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f008ca-1a5d-4226-8112-addcfdc2ebab service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:23.951015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196030c9-9f5f-44c7-9662-3db0d00760b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:23.951742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196030c9-9f5f-44c7-9662-3db0d00760b4 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:23.952154 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196030c9-9f5f-44c7-9662-3db0d00760b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:23.952566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196030c9-9f5f-44c7-9662-3db0d00760b4 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:23.953267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196030c9-9f5f-44c7-9662-3db0d00760b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:23.955375 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196030c9-9f5f-44c7-9662-3db0d00760b4 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:23.994009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 961/1961] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:23 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:23.997704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b3972-6a9d-4d8f-874e-5511c183aa90 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:24.010755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b3972-6a9d-4d8f-874e-5511c183aa90 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:24.025936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d283d98-b161-4c19-8d02-50a119264792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:24.026736 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d283d98-b161-4c19-8d02-50a119264792 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:24.027141 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d283d98-b161-4c19-8d02-50a119264792 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:24.027488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d283d98-b161-4c19-8d02-50a119264792 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:24.028281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d283d98-b161-4c19-8d02-50a119264792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:24.030440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d283d98-b161-4c19-8d02-50a119264792 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:24.050919 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1001/1962] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:24.055943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196030c9-9f5f-44c7-9662-3db0d00760b4 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:24.071732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196030c9-9f5f-44c7-9662-3db0d00760b4 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:24.079108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4ac66-0c23-4e5d-b1ca-8cea6c2db6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:24.080232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4ac66-0c23-4e5d-b1ca-8cea6c2db6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:24.080445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4ac66-0c23-4e5d-b1ca-8cea6c2db6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:24.080686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4ac66-0c23-4e5d-b1ca-8cea6c2db6fc admin admin] PATH_INFO: `/v3/projects/657bbcc599404bbeb4966588fbc46a10/users/03380e6fb869472c9f23024544e29d4b/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:24.103360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4ac66-0c23-4e5d-b1ca-8cea6c2db6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=657bbcc599404bbeb4966588fbc46a10, user_id=03380e6fb869472c9f23024544e29d4b, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:24.106293 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4ac66-0c23-4e5d-b1ca-8cea6c2db6fc admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:24.151868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 962/1963] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:24 2022] PUT /identity/v3/projects/657bbcc599404bbeb4966588fbc46a10/users/03380e6fb869472c9f23024544e29d4b/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:24.156710 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d283d98-b161-4c19-8d02-50a119264792 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:24.168756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d283d98-b161-4c19-8d02-50a119264792 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:24.243810 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc4089-543f-4971-af8a-a2af9e13457d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:24.244870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc4089-543f-4971-af8a-a2af9e13457d admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:24.245135 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc4089-543f-4971-af8a-a2af9e13457d admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:24.245374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc4089-543f-4971-af8a-a2af9e13457d admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:24.246172 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc4089-543f-4971-af8a-a2af9e13457d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:24.248290 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc4089-543f-4971-af8a-a2af9e13457d admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:24.260439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1002/1964] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:24.265033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4ac66-0c23-4e5d-b1ca-8cea6c2db6fc admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:24.277814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4ac66-0c23-4e5d-b1ca-8cea6c2db6fc admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:24.284698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1f004-1ce6-408a-88a7-68e8d72cafdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:24.285843 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1f004-1ce6-408a-88a7-68e8d72cafdc admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:24.286184 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1f004-1ce6-408a-88a7-68e8d72cafdc admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:24.286465 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1f004-1ce6-408a-88a7-68e8d72cafdc admin admin] PATH_INFO: `/v3/projects/657bbcc599404bbeb4966588fbc46a10/users/03380e6fb869472c9f23024544e29d4b/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:24.288951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1f004-1ce6-408a-88a7-68e8d72cafdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=657bbcc599404bbeb4966588fbc46a10, user_id=03380e6fb869472c9f23024544e29d4b, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:24.292157 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1f004-1ce6-408a-88a7-68e8d72cafdc admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:24.305533 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 963/1965] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:24 2022] PUT /identity/v3/projects/657bbcc599404bbeb4966588fbc46a10/users/03380e6fb869472c9f23024544e29d4b/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:24.315971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513d077-369d-4497-aad4-e44bab3f03e3 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:24.316281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513d077-369d-4497-aad4-e44bab3f03e3 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:24.316563 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513d077-369d-4497-aad4-e44bab3f03e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:24.352794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-5513d077-369d-4497-aad4-e44bab3f03e3 None None] MFA Rules not processed for user `03380e6fb869472c9f23024544e29d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:24.354240 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-5513d077-369d-4497-aad4-e44bab3f03e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:24.453521 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1003/1966] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:24.476360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1f004-1ce6-408a-88a7-68e8d72cafdc admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:24.586190 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1f004-1ce6-408a-88a7-68e8d72cafdc admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:24.592636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676ecae-6c1d-428b-8106-bf355f909b19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:24.593679 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676ecae-6c1d-428b-8106-bf355f909b19 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:24.594095 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676ecae-6c1d-428b-8106-bf355f909b19 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:24.594468 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676ecae-6c1d-428b-8106-bf355f909b19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:24.607378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676ecae-6c1d-428b-8106-bf355f909b19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:24.609581 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676ecae-6c1d-428b-8106-bf355f909b19 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:24.626396 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 964/1967] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:24.675525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513d077-369d-4497-aad4-e44bab3f03e3 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:24.678809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-5513d077-369d-4497-aad4-e44bab3f03e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:24.757218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676ecae-6c1d-428b-8106-bf355f909b19 service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:24.773431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676ecae-6c1d-428b-8106-bf355f909b19 service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:24.779589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc4a62-2886-49d6-b1ea-6fb67a33f4e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:24.781640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4a62-2886-49d6-b1ea-6fb67a33f4e8 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:24.787510 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4a62-2886-49d6-b1ea-6fb67a33f4e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:24.787681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4a62-2886-49d6-b1ea-6fb67a33f4e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:24.881508 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513d077-369d-4497-aad4-e44bab3f03e3 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:24.908158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77beef18-d956-4a65-97ce-fbff62617b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:24.909062 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77beef18-d956-4a65-97ce-fbff62617b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:24.910098 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77beef18-d956-4a65-97ce-fbff62617b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:24.910098 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77beef18-d956-4a65-97ce-fbff62617b34 admin admin] PATH_INFO: `/v3/users/2f99157ab7f94780a11fb74441adf632` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:24.912814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77beef18-d956-4a65-97ce-fbff62617b34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f99157ab7f94780a11fb74441adf632)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:24.914911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77beef18-d956-4a65-97ce-fbff62617b34 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:24.976033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc4a62-2886-49d6-b1ea-6fb67a33f4e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:24.977791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc4a62-2886-49d6-b1ea-6fb67a33f4e8 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:24.990176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 965/1968] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:30:25.023371 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-77beef18-d956-4a65-97ce-fbff62617b34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f99157ab7f94780a11fb74441adf632'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:25.024042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-77beef18-d956-4a65-97ce-fbff62617b34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f99157ab7f94780a11fb74441adf632'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:25.033604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-77beef18-d956-4a65-97ce-fbff62617b34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f99157ab7f94780a11fb74441adf632'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:25.044037 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-77beef18-d956-4a65-97ce-fbff62617b34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f99157ab7f94780a11fb74441adf632'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:25.045579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-77beef18-d956-4a65-97ce-fbff62617b34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f99157ab7f94780a11fb74441adf632'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:25.081001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1004/1969] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:24 2022] DELETE /identity/v3/users/2f99157ab7f94780a11fb74441adf632 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:25.085683 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59750b92-aa2a-4720-8659-9ea178e30da2 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:25.086581 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59750b92-aa2a-4720-8659-9ea178e30da2 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:25.087105 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59750b92-aa2a-4720-8659-9ea178e30da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:25.118675 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-59750b92-aa2a-4720-8659-9ea178e30da2 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:25.120345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-59750b92-aa2a-4720-8659-9ea178e30da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:25.219694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 966/1970] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:25.255701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77beef18-d956-4a65-97ce-fbff62617b34 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:25.257285 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-77beef18-d956-4a65-97ce-fbff62617b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:25.349082 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77beef18-d956-4a65-97ce-fbff62617b34 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:25.353674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a84b8-5041-4969-b340-9178454deca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:25.354435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a84b8-5041-4969-b340-9178454deca2 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:25.354826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a84b8-5041-4969-b340-9178454deca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:25.355208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a84b8-5041-4969-b340-9178454deca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:25.366544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a84b8-5041-4969-b340-9178454deca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:25.368306 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a84b8-5041-4969-b340-9178454deca2 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:25.379999 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1005/1971] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:25.868937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59750b92-aa2a-4720-8659-9ea178e30da2 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:25.870440 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-59750b92-aa2a-4720-8659-9ea178e30da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:25.883833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59750b92-aa2a-4720-8659-9ea178e30da2 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:25.889099 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee11c3-5fff-4d99-816f-a0475a565362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:25.890092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee11c3-5fff-4d99-816f-a0475a565362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:25.890838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee11c3-5fff-4d99-816f-a0475a565362 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:25.890838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee11c3-5fff-4d99-816f-a0475a565362 admin admin] PATH_INFO: `/v3/users/c1d626990b114bf3bd19bd6386831bb5` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:25.892062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee11c3-5fff-4d99-816f-a0475a565362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1d626990b114bf3bd19bd6386831bb5)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:25.894141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee11c3-5fff-4d99-816f-a0475a565362 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:25.941902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d3ee11c3-5fff-4d99-816f-a0475a565362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1d626990b114bf3bd19bd6386831bb5'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:25.948550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d3ee11c3-5fff-4d99-816f-a0475a565362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1d626990b114bf3bd19bd6386831bb5'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:25.949299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d3ee11c3-5fff-4d99-816f-a0475a565362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1d626990b114bf3bd19bd6386831bb5'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:25.961923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d3ee11c3-5fff-4d99-816f-a0475a565362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1d626990b114bf3bd19bd6386831bb5'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:25.961923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d3ee11c3-5fff-4d99-816f-a0475a565362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1d626990b114bf3bd19bd6386831bb5'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:25.980707 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 967/1972] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:25 2022] DELETE /identity/v3/users/c1d626990b114bf3bd19bd6386831bb5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:26.058330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a84b8-5041-4969-b340-9178454deca2 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:26.060622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-af9a84b8-5041-4969-b340-9178454deca2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:26.087220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a84b8-5041-4969-b340-9178454deca2 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:26.157621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:26.158536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:26.158818 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:26.159094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] PATH_INFO: `/v3/projects/071ef0d388954d70ad3129c58d81d756` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:26.160337 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=071ef0d388954d70ad3129c58d81d756)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:26.162229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:26.216061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '071ef0d388954d70ad3129c58d81d756', 'request_id': 'req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.223401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '071ef0d388954d70ad3129c58d81d756', 'request_id': 'req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.232105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '071ef0d388954d70ad3129c58d81d756', 'request_id': 'req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.236805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee11c3-5fff-4d99-816f-a0475a565362 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:26.238605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-d3ee11c3-5fff-4d99-816f-a0475a565362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:26.255999 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] The token cache is being invalidate because project 071ef0d388954d70ad3129c58d81d756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:26.256944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.257849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.260023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1006/1973] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:26 2022] DELETE /identity/v3/projects/071ef0d388954d70ad3129c58d81d756 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:26.261027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee11c3-5fff-4d99-816f-a0475a565362 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:26.269571 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:26.271113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:26.326375 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:26.328073 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:26.328542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:26.329026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] PATH_INFO: `/v3/projects/df9194d5f9174b7c943c71fa7bfb3318` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:26.330224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df9194d5f9174b7c943c71fa7bfb3318)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:26.332196 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:26.349542 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228632d-9e9b-4f76-8e2d-54ffc6ff90aa admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:26.362230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ce55b-fc20-4c18-94de-02dc2650cbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:26.363459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ce55b-fc20-4c18-94de-02dc2650cbac admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:26.364020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ce55b-fc20-4c18-94de-02dc2650cbac admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:26.366034 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ce55b-fc20-4c18-94de-02dc2650cbac admin admin] PATH_INFO: `/v3/users/42d1f33168484b35a5c26959a66d2f40` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:26.367551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ce55b-fc20-4c18-94de-02dc2650cbac admin admin] RBAC: Authorizing `identity:delete_user(user_id=42d1f33168484b35a5c26959a66d2f40)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:26.369700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ce55b-fc20-4c18-94de-02dc2650cbac admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:26.393489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df9194d5f9174b7c943c71fa7bfb3318', 'request_id': 'req-2ab6b70f-392d-4f9b-a1c1-20b17491e171'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.402788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df9194d5f9174b7c943c71fa7bfb3318', 'request_id': 'req-2ab6b70f-392d-4f9b-a1c1-20b17491e171'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.411386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df9194d5f9174b7c943c71fa7bfb3318', 'request_id': 'req-2ab6b70f-392d-4f9b-a1c1-20b17491e171'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.426849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] The token cache is being invalidate because project df9194d5f9174b7c943c71fa7bfb3318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:26.427569 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.429079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.435476 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 968/1974] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:26 2022] DELETE /identity/v3/projects/df9194d5f9174b7c943c71fa7bfb3318 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:26.436347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:26.437808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:26.441792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f26ce55b-fc20-4c18-94de-02dc2650cbac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42d1f33168484b35a5c26959a66d2f40'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.442473 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f26ce55b-fc20-4c18-94de-02dc2650cbac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42d1f33168484b35a5c26959a66d2f40'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.454129 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f26ce55b-fc20-4c18-94de-02dc2650cbac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42d1f33168484b35a5c26959a66d2f40'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.463629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f26ce55b-fc20-4c18-94de-02dc2650cbac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42d1f33168484b35a5c26959a66d2f40'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.464287 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f26ce55b-fc20-4c18-94de-02dc2650cbac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42d1f33168484b35a5c26959a66d2f40'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.487726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1007/1975] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:26 2022] DELETE /identity/v3/users/42d1f33168484b35a5c26959a66d2f40 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:26.517763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6b70f-392d-4f9b-a1c1-20b17491e171 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:26.594266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d50009-8fe6-4a84-a24b-f55dfdfc718f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:26.595072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d50009-8fe6-4a84-a24b-f55dfdfc718f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:26.595380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d50009-8fe6-4a84-a24b-f55dfdfc718f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:26.596171 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d50009-8fe6-4a84-a24b-f55dfdfc718f admin admin] PATH_INFO: `/v3/users/7ee8d752027e4d14b7344ec32b95fb4e` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:26.598022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d50009-8fe6-4a84-a24b-f55dfdfc718f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ee8d752027e4d14b7344ec32b95fb4e)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:26.601411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d50009-8fe6-4a84-a24b-f55dfdfc718f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:26.676824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-85d50009-8fe6-4a84-a24b-f55dfdfc718f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ee8d752027e4d14b7344ec32b95fb4e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.698512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-85d50009-8fe6-4a84-a24b-f55dfdfc718f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ee8d752027e4d14b7344ec32b95fb4e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.699429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-85d50009-8fe6-4a84-a24b-f55dfdfc718f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ee8d752027e4d14b7344ec32b95fb4e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.712470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-85d50009-8fe6-4a84-a24b-f55dfdfc718f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ee8d752027e4d14b7344ec32b95fb4e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.713159 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-85d50009-8fe6-4a84-a24b-f55dfdfc718f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ee8d752027e4d14b7344ec32b95fb4e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:26.772353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 969/1976] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:26 2022] DELETE /identity/v3/users/7ee8d752027e4d14b7344ec32b95fb4e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:27.321510 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ce55b-fc20-4c18-94de-02dc2650cbac admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:27.323032 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f26ce55b-fc20-4c18-94de-02dc2650cbac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:27.355547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ce55b-fc20-4c18-94de-02dc2650cbac admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:27.545483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:27.545483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:27.545483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:27.545483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] PATH_INFO: `/v3/projects/ea22e236a132454591b3a93c707554ab` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:27.548359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea22e236a132454591b3a93c707554ab)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:27.550325 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:27.635301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea22e236a132454591b3a93c707554ab', 'request_id': 'req-37964460-1bf8-4e1b-902b-756c79c6eb4a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.642479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d50009-8fe6-4a84-a24b-f55dfdfc718f admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:27.645626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-85d50009-8fe6-4a84-a24b-f55dfdfc718f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:27.651415 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea22e236a132454591b3a93c707554ab', 'request_id': 'req-37964460-1bf8-4e1b-902b-756c79c6eb4a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.660227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea22e236a132454591b3a93c707554ab', 'request_id': 'req-37964460-1bf8-4e1b-902b-756c79c6eb4a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.670647 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d50009-8fe6-4a84-a24b-f55dfdfc718f admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:27.679506 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] The token cache is being invalidate because project ea22e236a132454591b3a93c707554ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:27.680487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.683727 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.686798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1008/1977] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:27 2022] DELETE /identity/v3/projects/ea22e236a132454591b3a93c707554ab => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:27.690879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:27.693496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:27.767941 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eb412-cc3b-41b5-8659-f9ced6f3004b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:27.768830 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eb412-cc3b-41b5-8659-f9ced6f3004b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:27.769115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eb412-cc3b-41b5-8659-f9ced6f3004b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:27.769395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eb412-cc3b-41b5-8659-f9ced6f3004b admin admin] PATH_INFO: `/v3/projects/3a7871c956d24192801f52ee5e3692e8` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:27.770966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06eb412-cc3b-41b5-8659-f9ced6f3004b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a7871c956d24192801f52ee5e3692e8)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:27.773357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06eb412-cc3b-41b5-8659-f9ced6f3004b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:27.779889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37964460-1bf8-4e1b-902b-756c79c6eb4a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:27.786821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e188e7b-d2a8-4d58-b501-e8c9bb630f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:27.787765 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e188e7b-d2a8-4d58-b501-e8c9bb630f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:27.789314 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e188e7b-d2a8-4d58-b501-e8c9bb630f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:27.789599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e188e7b-d2a8-4d58-b501-e8c9bb630f8d admin admin] PATH_INFO: `/v3/users/b0d1720b2a2d4c269ae68c0c01dc70ac` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:27.791832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e188e7b-d2a8-4d58-b501-e8c9bb630f8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0d1720b2a2d4c269ae68c0c01dc70ac)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:27.793767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e188e7b-d2a8-4d58-b501-e8c9bb630f8d admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:27.846171 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a06eb412-cc3b-41b5-8659-f9ced6f3004b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a7871c956d24192801f52ee5e3692e8', 'request_id': 'req-a06eb412-cc3b-41b5-8659-f9ced6f3004b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.855425 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a06eb412-cc3b-41b5-8659-f9ced6f3004b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a7871c956d24192801f52ee5e3692e8', 'request_id': 'req-a06eb412-cc3b-41b5-8659-f9ced6f3004b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.865068 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a06eb412-cc3b-41b5-8659-f9ced6f3004b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a7871c956d24192801f52ee5e3692e8', 'request_id': 'req-a06eb412-cc3b-41b5-8659-f9ced6f3004b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.881532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a06eb412-cc3b-41b5-8659-f9ced6f3004b admin admin] The token cache is being invalidate because project 3a7871c956d24192801f52ee5e3692e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:27.881532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a06eb412-cc3b-41b5-8659-f9ced6f3004b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.881532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a06eb412-cc3b-41b5-8659-f9ced6f3004b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.882670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 970/1978] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:27 2022] DELETE /identity/v3/projects/3a7871c956d24192801f52ee5e3692e8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:27.888889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d26ea-de48-461c-aa13-0e8a05e1ec83 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:27.889863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d26ea-de48-461c-aa13-0e8a05e1ec83 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:27.890159 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d26ea-de48-461c-aa13-0e8a05e1ec83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:27.890719 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2e188e7b-d2a8-4d58-b501-e8c9bb630f8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0d1720b2a2d4c269ae68c0c01dc70ac'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.891699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2e188e7b-d2a8-4d58-b501-e8c9bb630f8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0d1720b2a2d4c269ae68c0c01dc70ac'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.910415 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2e188e7b-d2a8-4d58-b501-e8c9bb630f8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0d1720b2a2d4c269ae68c0c01dc70ac'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.933935 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2e188e7b-d2a8-4d58-b501-e8c9bb630f8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0d1720b2a2d4c269ae68c0c01dc70ac'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.934352 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2e188e7b-d2a8-4d58-b501-e8c9bb630f8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0d1720b2a2d4c269ae68c0c01dc70ac'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:27.946403 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ce5d26ea-de48-461c-aa13-0e8a05e1ec83 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:27.948285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ce5d26ea-de48-461c-aa13-0e8a05e1ec83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:27.986179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1009/1979] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:27 2022] DELETE /identity/v3/users/b0d1720b2a2d4c269ae68c0c01dc70ac => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:28.052636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 971/1980] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:28.061877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff681226-4243-4287-a72b-2fa516ab94e8 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:28.061877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff681226-4243-4287-a72b-2fa516ab94e8 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:28.061877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff681226-4243-4287-a72b-2fa516ab94e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:28.110641 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-ff681226-4243-4287-a72b-2fa516ab94e8 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:28.114321 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ff681226-4243-4287-a72b-2fa516ab94e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:28.232440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1010/1981] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:28.237762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d26ea-de48-461c-aa13-0e8a05e1ec83 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:28.252832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d26ea-de48-461c-aa13-0e8a05e1ec83 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:28.262070 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ef66d-638d-47e6-8892-5bbc4e430f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:28.262757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ef66d-638d-47e6-8892-5bbc4e430f7a admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:28.265717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ef66d-638d-47e6-8892-5bbc4e430f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:28.266006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ef66d-638d-47e6-8892-5bbc4e430f7a admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:28.271305 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ef66d-638d-47e6-8892-5bbc4e430f7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:30:28.271620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ef66d-638d-47e6-8892-5bbc4e430f7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:28.273372 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ef66d-638d-47e6-8892-5bbc4e430f7a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:28.289523 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 972/1982] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:30:28.294552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff681226-4243-4287-a72b-2fa516ab94e8 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:28.313949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff681226-4243-4287-a72b-2fa516ab94e8 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:28.324681 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e3fe8-19ae-4bb0-94b3-bd18f0a6a8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:28.326139 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e3fe8-19ae-4bb0-94b3-bd18f0a6a8af admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:28.326861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e3fe8-19ae-4bb0-94b3-bd18f0a6a8af admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:28.327527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e3fe8-19ae-4bb0-94b3-bd18f0a6a8af admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:28.328555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e3fe8-19ae-4bb0-94b3-bd18f0a6a8af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:28.330769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e3fe8-19ae-4bb0-94b3-bd18f0a6a8af admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:28.390622 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1011/1983] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:28 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:28.395569 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ef66d-638d-47e6-8892-5bbc4e430f7a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:28.414764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ef66d-638d-47e6-8892-5bbc4e430f7a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:28.530721 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b7de2-59c0-4a34-bfce-5ec992ff34c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:28.531718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b7de2-59c0-4a34-bfce-5ec992ff34c5 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:28.532445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b7de2-59c0-4a34-bfce-5ec992ff34c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:28.532798 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b7de2-59c0-4a34-bfce-5ec992ff34c5 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:28.534446 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b7de2-59c0-4a34-bfce-5ec992ff34c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:28.545619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b7de2-59c0-4a34-bfce-5ec992ff34c5 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:28.594074 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 973/1984] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:28 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:28.600574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e3fe8-19ae-4bb0-94b3-bd18f0a6a8af admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:28.625708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b7de2-59c0-4a34-bfce-5ec992ff34c5 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:28.628819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-da4b7de2-59c0-4a34-bfce-5ec992ff34c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:28.633182 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e3fe8-19ae-4bb0-94b3-bd18f0a6a8af admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:28.661084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c7051-6b0d-4fa5-aaff-fd8c4ab787b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:28.662219 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c7051-6b0d-4fa5-aaff-fd8c4ab787b6 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:28.662874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c7051-6b0d-4fa5-aaff-fd8c4ab787b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:28.663502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c7051-6b0d-4fa5-aaff-fd8c4ab787b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:28.670852 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c7051-6b0d-4fa5-aaff-fd8c4ab787b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:28.673718 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c7051-6b0d-4fa5-aaff-fd8c4ab787b6 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:28.679080 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b7de2-59c0-4a34-bfce-5ec992ff34c5 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:28.707379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:28.708542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:28.708827 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:28.709104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] PATH_INFO: `/v3/projects/6895f7368f3a4472b18b7d2d27943209` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:28.710155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1012/1985] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:28.711296 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6895f7368f3a4472b18b7d2d27943209)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:28.717406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c7051-6b0d-4fa5-aaff-fd8c4ab787b6 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:28.719984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:28.734480 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c7051-6b0d-4fa5-aaff-fd8c4ab787b6 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:28.749608 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff283957-fc0a-4429-aaeb-e40badd65733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:28.750434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff283957-fc0a-4429-aaeb-e40badd65733 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:28.750955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff283957-fc0a-4429-aaeb-e40badd65733 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:28.751479 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff283957-fc0a-4429-aaeb-e40badd65733 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:28.752674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff283957-fc0a-4429-aaeb-e40badd65733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:28.758136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff283957-fc0a-4429-aaeb-e40badd65733 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:28.784754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1013/1986] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:28.792701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff283957-fc0a-4429-aaeb-e40badd65733 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:28.809050 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff283957-fc0a-4429-aaeb-e40badd65733 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:28.814628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6895f7368f3a4472b18b7d2d27943209', 'request_id': 'req-618631b6-0b53-4b97-b0a6-8d5ead5be87e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:28.846896 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6895f7368f3a4472b18b7d2d27943209', 'request_id': 'req-618631b6-0b53-4b97-b0a6-8d5ead5be87e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:28.873383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6895f7368f3a4472b18b7d2d27943209', 'request_id': 'req-618631b6-0b53-4b97-b0a6-8d5ead5be87e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:28.898451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] The token cache is being invalidate because project 6895f7368f3a4472b18b7d2d27943209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:28.898898 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:28.901459 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:28.905008 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 974/1987] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:28 2022] DELETE /identity/v3/projects/6895f7368f3a4472b18b7d2d27943209 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:28.906085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:28.909548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:29.020124 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e1e70-9ef4-43c1-ac75-b626a05828a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:29.021041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e1e70-9ef4-43c1-ac75-b626a05828a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.021323 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e1e70-9ef4-43c1-ac75-b626a05828a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.021600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e1e70-9ef4-43c1-ac75-b626a05828a4 admin admin] PATH_INFO: `/v3/projects/63886a179df746d4b2a750f8054028cf/users/75b86c0021e7411b9e4f7cc1e65b4f2b/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.022123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618631b6-0b53-4b97-b0a6-8d5ead5be87e admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:29.034079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75642697-51a4-4e8c-9e3f-8db4e698be0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:29.035064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75642697-51a4-4e8c-9e3f-8db4e698be0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.035610 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75642697-51a4-4e8c-9e3f-8db4e698be0b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.036007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75642697-51a4-4e8c-9e3f-8db4e698be0b admin admin] PATH_INFO: `/v3/users/c322a8ad6388470c99b0248e938d8316` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.037582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75642697-51a4-4e8c-9e3f-8db4e698be0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c322a8ad6388470c99b0248e938d8316)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:29.040558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75642697-51a4-4e8c-9e3f-8db4e698be0b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:29.041166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e1e70-9ef4-43c1-ac75-b626a05828a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63886a179df746d4b2a750f8054028cf, user_id=75b86c0021e7411b9e4f7cc1e65b4f2b, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:29.042866 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e1e70-9ef4-43c1-ac75-b626a05828a4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:29.069698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1014/1988] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:28 2022] PUT /identity/v3/projects/63886a179df746d4b2a750f8054028cf/users/75b86c0021e7411b9e4f7cc1e65b4f2b/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:29.069698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b5c5a-5c11-41fe-bc7d-e99eef8eb98c None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.069698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b5c5a-5c11-41fe-bc7d-e99eef8eb98c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.069698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b5c5a-5c11-41fe-bc7d-e99eef8eb98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.098627 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-7c0b5c5a-5c11-41fe-bc7d-e99eef8eb98c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:29.099891 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7c0b5c5a-5c11-41fe-bc7d-e99eef8eb98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:29.116200 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-75642697-51a4-4e8c-9e3f-8db4e698be0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c322a8ad6388470c99b0248e938d8316'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:29.130354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-75642697-51a4-4e8c-9e3f-8db4e698be0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c322a8ad6388470c99b0248e938d8316'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:29.131354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-75642697-51a4-4e8c-9e3f-8db4e698be0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c322a8ad6388470c99b0248e938d8316'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:29.141233 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-75642697-51a4-4e8c-9e3f-8db4e698be0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c322a8ad6388470c99b0248e938d8316'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:29.142349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-75642697-51a4-4e8c-9e3f-8db4e698be0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c322a8ad6388470c99b0248e938d8316'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:29.171139 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 975/1989] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:28 2022] DELETE /identity/v3/users/c322a8ad6388470c99b0248e938d8316 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:29.173061 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde65351-d104-4282-b998-dbea98d60076 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.173527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde65351-d104-4282-b998-dbea98d60076 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.173716 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde65351-d104-4282-b998-dbea98d60076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.182127 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1015/1990] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:29.183333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1c997-c67b-4854-b441-3fa0ec4955b0 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.183630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1c997-c67b-4854-b441-3fa0ec4955b0 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.183910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1c997-c67b-4854-b441-3fa0ec4955b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.205410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2eb1c997-c67b-4854-b441-3fa0ec4955b0 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:29.207816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2eb1c997-c67b-4854-b441-3fa0ec4955b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:29.212825 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-fde65351-d104-4282-b998-dbea98d60076 None None] MFA Rules not processed for user `75b86c0021e7411b9e4f7cc1e65b4f2b`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:29.215014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-fde65351-d104-4282-b998-dbea98d60076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:29.296495 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1016/1991] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:29.298039 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357264b6-c810-456c-8819-c612b2cee859 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.298330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357264b6-c810-456c-8819-c612b2cee859 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.298603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357264b6-c810-456c-8819-c612b2cee859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.306660 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 976/1992] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:29.311253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde65351-d104-4282-b998-dbea98d60076 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:29.313437 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-fde65351-d104-4282-b998-dbea98d60076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:29.327095 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-357264b6-c810-456c-8819-c612b2cee859 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:29.328330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-357264b6-c810-456c-8819-c612b2cee859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:29.337365 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde65351-d104-4282-b998-dbea98d60076 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:29.341443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1017/1993] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:29.342572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357264b6-c810-456c-8819-c612b2cee859 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:29.344708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-357264b6-c810-456c-8819-c612b2cee859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:29.348145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe931a-857f-4b6e-8638-6dd8a121672e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:29.349092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe931a-857f-4b6e-8638-6dd8a121672e admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.349384 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe931a-857f-4b6e-8638-6dd8a121672e admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.349820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe931a-857f-4b6e-8638-6dd8a121672e admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.350624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe931a-857f-4b6e-8638-6dd8a121672e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:29.353125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe931a-857f-4b6e-8638-6dd8a121672e admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:29.394689 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 977/1994] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:29 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:30:29.396671 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe931a-857f-4b6e-8638-6dd8a121672e admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:29.421662 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357264b6-c810-456c-8819-c612b2cee859 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:29.477892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe931a-857f-4b6e-8638-6dd8a121672e admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:29.483683 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decbf56d-0aa8-409f-9a20-a26495cac459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:29.484370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbf56d-0aa8-409f-9a20-a26495cac459 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.484659 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbf56d-0aa8-409f-9a20-a26495cac459 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.484934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbf56d-0aa8-409f-9a20-a26495cac459 admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.485664 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decbf56d-0aa8-409f-9a20-a26495cac459 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:30:29.485980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decbf56d-0aa8-409f-9a20-a26495cac459 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:29.487755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decbf56d-0aa8-409f-9a20-a26495cac459 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:29.493147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8642d8-af30-4e40-97a9-f558fe5f0f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:29.493870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8642d8-af30-4e40-97a9-f558fe5f0f77 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.494169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8642d8-af30-4e40-97a9-f558fe5f0f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.494473 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8642d8-af30-4e40-97a9-f558fe5f0f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.500872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 978/1995] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:30:29.500872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decbf56d-0aa8-409f-9a20-a26495cac459 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:29.506422 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8642d8-af30-4e40-97a9-f558fe5f0f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:29.507893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8642d8-af30-4e40-97a9-f558fe5f0f77 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:29.517693 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decbf56d-0aa8-409f-9a20-a26495cac459 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:29.520498 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1018/1996] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:29.521531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8642d8-af30-4e40-97a9-f558fe5f0f77 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:29.525271 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9b80b-ab5b-4337-b2bb-a52e48be510e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:29.529789 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9b80b-ab5b-4337-b2bb-a52e48be510e admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.530501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9b80b-ab5b-4337-b2bb-a52e48be510e admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.531089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9b80b-ab5b-4337-b2bb-a52e48be510e admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.532013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9b80b-ab5b-4337-b2bb-a52e48be510e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:29.533115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8642d8-af30-4e40-97a9-f558fe5f0f77 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:29.534439 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9b80b-ab5b-4337-b2bb-a52e48be510e admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:29.541987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a02d5c-13dd-421e-98f2-59ccdabeaad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:29.544849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a02d5c-13dd-421e-98f2-59ccdabeaad4 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.544849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a02d5c-13dd-421e-98f2-59ccdabeaad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.544849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a02d5c-13dd-421e-98f2-59ccdabeaad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.544849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a02d5c-13dd-421e-98f2-59ccdabeaad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:29.546808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a02d5c-13dd-421e-98f2-59ccdabeaad4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:29.578251 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 979/1997] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:29 2022] POST /identity/v3/users => generated 299 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:29.581433 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9b80b-ab5b-4337-b2bb-a52e48be510e admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:29.589193 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1019/1998] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:29 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:29.592652 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a02d5c-13dd-421e-98f2-59ccdabeaad4 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:29.597832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9b80b-ab5b-4337-b2bb-a52e48be510e admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:29.706103 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f4bfa-4f41-432d-9d3d-1bb20a9e9da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:29.706910 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f4bfa-4f41-432d-9d3d-1bb20a9e9da2 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.707222 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f4bfa-4f41-432d-9d3d-1bb20a9e9da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.707763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f4bfa-4f41-432d-9d3d-1bb20a9e9da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.708303 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f4bfa-4f41-432d-9d3d-1bb20a9e9da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:29.708969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a02d5c-13dd-421e-98f2-59ccdabeaad4 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:29.710017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f4bfa-4f41-432d-9d3d-1bb20a9e9da2 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:29.719347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3f4a1-eec1-4371-8840-929168274e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:29.720436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3f4a1-eec1-4371-8840-929168274e02 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.723360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3f4a1-eec1-4371-8840-929168274e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.723904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3f4a1-eec1-4371-8840-929168274e02 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.727130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 980/1999] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:29.728191 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f4bfa-4f41-432d-9d3d-1bb20a9e9da2 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:29.730715 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3f4a1-eec1-4371-8840-929168274e02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:29.737431 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3f4a1-eec1-4371-8840-929168274e02 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:29.746195 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f4bfa-4f41-432d-9d3d-1bb20a9e9da2 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:29.752745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cce49-caca-440d-b527-5871dadef883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:29.754074 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cce49-caca-440d-b527-5871dadef883 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.754412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cce49-caca-440d-b527-5871dadef883 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.755015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cce49-caca-440d-b527-5871dadef883 admin admin] PATH_INFO: `/v3/projects/542388eeb32d4227bd4ae743d7e9dc14/users/a733d683dbd347648aacb5062a755f0c/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.780140 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cce49-caca-440d-b527-5871dadef883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=542388eeb32d4227bd4ae743d7e9dc14, user_id=a733d683dbd347648aacb5062a755f0c, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:29.783670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cce49-caca-440d-b527-5871dadef883 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:29.801437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1020/2000] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:29 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:29.805454 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 981/2001] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:29 2022] PUT /identity/v3/projects/542388eeb32d4227bd4ae743d7e9dc14/users/a733d683dbd347648aacb5062a755f0c/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:29.806733 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3f4a1-eec1-4371-8840-929168274e02 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:29.808661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cce49-caca-440d-b527-5871dadef883 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:29.827829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3f4a1-eec1-4371-8840-929168274e02 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:29.920370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cce49-caca-440d-b527-5871dadef883 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:29.936215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb4774-fcd4-4a98-9b23-d28a7f0dbf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:29.938238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb4774-fcd4-4a98-9b23-d28a7f0dbf1a admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.939002 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb4774-fcd4-4a98-9b23-d28a7f0dbf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.939120 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb4774-fcd4-4a98-9b23-d28a7f0dbf1a admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.939904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb4774-fcd4-4a98-9b23-d28a7f0dbf1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:29.942466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb4774-fcd4-4a98-9b23-d28a7f0dbf1a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:29.959098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 982/2002] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:29.960166 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb4774-fcd4-4a98-9b23-d28a7f0dbf1a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:29.961998 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6abca1-251f-4b03-b947-62e6bf964e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:29.962714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6abca1-251f-4b03-b947-62e6bf964e08 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.963052 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6abca1-251f-4b03-b947-62e6bf964e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.963409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6abca1-251f-4b03-b947-62e6bf964e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.964185 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6abca1-251f-4b03-b947-62e6bf964e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:29.965835 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6abca1-251f-4b03-b947-62e6bf964e08 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:29.975721 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb4774-fcd4-4a98-9b23-d28a7f0dbf1a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:29.982623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d7f20-665a-4133-a9b1-2a1005abf0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:29.983352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d7f20-665a-4133-a9b1-2a1005abf0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:29.983641 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d7f20-665a-4133-a9b1-2a1005abf0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:29.983922 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d7f20-665a-4133-a9b1-2a1005abf0c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:29.984859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d7f20-665a-4133-a9b1-2a1005abf0c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:29.986685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d7f20-665a-4133-a9b1-2a1005abf0c5 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:29.990970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1021/2003] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:30:29.996107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6abca1-251f-4b03-b947-62e6bf964e08 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:30.006381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 983/2004] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:30.013697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d7f20-665a-4133-a9b1-2a1005abf0c5 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:30.033062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d7f20-665a-4133-a9b1-2a1005abf0c5 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:30.035626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6abca1-251f-4b03-b947-62e6bf964e08 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:30.054602 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d0d22-e174-4460-afc2-76263bcfeecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:30.055705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d0d22-e174-4460-afc2-76263bcfeecf admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:30.056146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d0d22-e174-4460-afc2-76263bcfeecf admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:30.056513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d0d22-e174-4460-afc2-76263bcfeecf admin admin] PATH_INFO: `/v3/projects/22b5c7e0342346b9baa3f4d8a77a6fef/users/ab362ab34eba451ea7bb2fdbc27a6030/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:30.062573 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b35ba-45b7-4f34-a448-7df0d0b0244b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:30.064849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b35ba-45b7-4f34-a448-7df0d0b0244b admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:30.066029 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b35ba-45b7-4f34-a448-7df0d0b0244b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:30.066498 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b35ba-45b7-4f34-a448-7df0d0b0244b admin admin] PATH_INFO: `/v3/projects/542388eeb32d4227bd4ae743d7e9dc14/users/a733d683dbd347648aacb5062a755f0c/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:30.072908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b35ba-45b7-4f34-a448-7df0d0b0244b admin admin] RBAC: Authorizing `identity:create_grant(project_id=542388eeb32d4227bd4ae743d7e9dc14, user_id=a733d683dbd347648aacb5062a755f0c, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:30.076458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b35ba-45b7-4f34-a448-7df0d0b0244b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:30.084943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d0d22-e174-4460-afc2-76263bcfeecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=22b5c7e0342346b9baa3f4d8a77a6fef, user_id=ab362ab34eba451ea7bb2fdbc27a6030, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:30.087871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d0d22-e174-4460-afc2-76263bcfeecf admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:30.108268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1022/2005] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:29 2022] PUT /identity/v3/projects/542388eeb32d4227bd4ae743d7e9dc14/users/a733d683dbd347648aacb5062a755f0c/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:30.109558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b35ba-45b7-4f34-a448-7df0d0b0244b admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:30.110631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 984/2006] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:30 2022] PUT /identity/v3/projects/22b5c7e0342346b9baa3f4d8a77a6fef/users/ab362ab34eba451ea7bb2fdbc27a6030/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:30.116836 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-533b35ba-45b7-4f34-a448-7df0d0b0244b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:30.123803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353af652-1d6d-411a-af47-9eb4e4be8265 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:30.124125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353af652-1d6d-411a-af47-9eb4e4be8265 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:30.124361 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353af652-1d6d-411a-af47-9eb4e4be8265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:30.183876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-353af652-1d6d-411a-af47-9eb4e4be8265 None None] MFA Rules not processed for user `a733d683dbd347648aacb5062a755f0c`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:30.185747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-353af652-1d6d-411a-af47-9eb4e4be8265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:30.218036 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b35ba-45b7-4f34-a448-7df0d0b0244b admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:30.306711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 985/2007] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:30.312588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c818955-cdd9-4225-b325-bc04e067ee87 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:30.312959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c818955-cdd9-4225-b325-bc04e067ee87 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:30.313392 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c818955-cdd9-4225-b325-bc04e067ee87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:30.317177 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:30.318239 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:30.318687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:30.319103 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] PATH_INFO: `/v3/projects/4c6dbd06737d4b8b84a3d09e929e9eb8` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:30.321455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c6dbd06737d4b8b84a3d09e929e9eb8)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:30.324041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:30.347728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-8c818955-cdd9-4225-b325-bc04e067ee87 None None] MFA Rules not processed for user `ab362ab34eba451ea7bb2fdbc27a6030`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:30.348956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-8c818955-cdd9-4225-b325-bc04e067ee87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:30.384140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c6dbd06737d4b8b84a3d09e929e9eb8', 'request_id': 'req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:30.393952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c6dbd06737d4b8b84a3d09e929e9eb8', 'request_id': 'req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:30.406132 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c6dbd06737d4b8b84a3d09e929e9eb8', 'request_id': 'req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:30.420512 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] The token cache is being invalidate because project 4c6dbd06737d4b8b84a3d09e929e9eb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:30.420966 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:30.421681 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:30.423967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1023/2008] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:30 2022] DELETE /identity/v3/projects/4c6dbd06737d4b8b84a3d09e929e9eb8 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:30.425247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:30.427140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:30.440109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 986/2009] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:30.441378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c818955-cdd9-4225-b325-bc04e067ee87 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:30.460329 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c818955-cdd9-4225-b325-bc04e067ee87 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:30.498511 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba03a6c-3d5e-4546-b50e-6500a52dab7b admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:30.503888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3dfaf-8475-42ab-86f7-3762f110c439 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:30.507737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3dfaf-8475-42ab-86f7-3762f110c439 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:30.508122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3dfaf-8475-42ab-86f7-3762f110c439 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:30.508471 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3dfaf-8475-42ab-86f7-3762f110c439 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:30.511518 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-eeb3dfaf-8475-42ab-86f7-3762f110c439 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:30.549167 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad01ad3-acf5-429d-86f2-7596815c3b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:30.551650 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad01ad3-acf5-429d-86f2-7596815c3b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:30.553025 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad01ad3-acf5-429d-86f2-7596815c3b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:30.553523 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad01ad3-acf5-429d-86f2-7596815c3b13 admin admin] PATH_INFO: `/v3/users/1c7e202e69d5412ab61d72654791dcac` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:30.557023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad01ad3-acf5-429d-86f2-7596815c3b13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c7e202e69d5412ab61d72654791dcac)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:30.560375 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad01ad3-acf5-429d-86f2-7596815c3b13 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:30.594208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3dfaf-8475-42ab-86f7-3762f110c439 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:30.595723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3dfaf-8475-42ab-86f7-3762f110c439 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:30.610415 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1024/2010] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:30.611510 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3dfaf-8475-42ab-86f7-3762f110c439 service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:30.612834 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-eeb3dfaf-8475-42ab-86f7-3762f110c439 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:30.625771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3dfaf-8475-42ab-86f7-3762f110c439 service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:30.631691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729dca9d-5587-4eaf-b69e-35b2163ae079 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:30.632367 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dca9d-5587-4eaf-b69e-35b2163ae079 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:30.632984 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dca9d-5587-4eaf-b69e-35b2163ae079 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:30.633273 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dca9d-5587-4eaf-b69e-35b2163ae079 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:30.636625 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-5ad01ad3-acf5-429d-86f2-7596815c3b13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c7e202e69d5412ab61d72654791dcac'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:30.637023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-729dca9d-5587-4eaf-b69e-35b2163ae079 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:30.651898 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-5ad01ad3-acf5-429d-86f2-7596815c3b13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c7e202e69d5412ab61d72654791dcac'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:30.654127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-5ad01ad3-acf5-429d-86f2-7596815c3b13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c7e202e69d5412ab61d72654791dcac'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:30.675529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-5ad01ad3-acf5-429d-86f2-7596815c3b13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c7e202e69d5412ab61d72654791dcac'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:30.678934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-5ad01ad3-acf5-429d-86f2-7596815c3b13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c7e202e69d5412ab61d72654791dcac'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:30.704256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 987/2011] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:30 2022] DELETE /identity/v3/users/1c7e202e69d5412ab61d72654791dcac => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:30.744080 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729dca9d-5587-4eaf-b69e-35b2163ae079 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:30.746760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729dca9d-5587-4eaf-b69e-35b2163ae079 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:30.774268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1025/2012] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:30.806332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad01ad3-acf5-429d-86f2-7596815c3b13 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:30.809366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-5ad01ad3-acf5-429d-86f2-7596815c3b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:30.932785 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad01ad3-acf5-429d-86f2-7596815c3b13 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:30.944800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08da565-3648-4501-bfe6-230914dbec6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:30.944800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08da565-3648-4501-bfe6-230914dbec6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:30.945128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08da565-3648-4501-bfe6-230914dbec6f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:30.945346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08da565-3648-4501-bfe6-230914dbec6f admin admin] PATH_INFO: `/v3/users/ab362ab34eba451ea7bb2fdbc27a6030` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:30.947281 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08da565-3648-4501-bfe6-230914dbec6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab362ab34eba451ea7bb2fdbc27a6030)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:30.957790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08da565-3648-4501-bfe6-230914dbec6f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:31.059381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c08da565-3648-4501-bfe6-230914dbec6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab362ab34eba451ea7bb2fdbc27a6030'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:31.078235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c08da565-3648-4501-bfe6-230914dbec6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab362ab34eba451ea7bb2fdbc27a6030'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:31.079033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c08da565-3648-4501-bfe6-230914dbec6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab362ab34eba451ea7bb2fdbc27a6030'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:31.094282 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c08da565-3648-4501-bfe6-230914dbec6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab362ab34eba451ea7bb2fdbc27a6030'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:31.096119 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c08da565-3648-4501-bfe6-230914dbec6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab362ab34eba451ea7bb2fdbc27a6030'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:31.130321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 988/2013] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:30 2022] DELETE /identity/v3/users/ab362ab34eba451ea7bb2fdbc27a6030 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:31.136303 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dded059-8b6e-4e60-9922-56cca3b1597b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:31.136620 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dded059-8b6e-4e60-9922-56cca3b1597b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:31.136906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dded059-8b6e-4e60-9922-56cca3b1597b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:31.168750 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4dded059-8b6e-4e60-9922-56cca3b1597b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:31.172319 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4dded059-8b6e-4e60-9922-56cca3b1597b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:31.186735 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08da565-3648-4501-bfe6-230914dbec6f admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:31.188458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c08da565-3648-4501-bfe6-230914dbec6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:31.346254 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08da565-3648-4501-bfe6-230914dbec6f admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:31.350205 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1026/2014] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:31.355815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ab425-f9fe-448d-b4c3-d5282b2febfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:31.356742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ab425-f9fe-448d-b4c3-d5282b2febfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:31.357118 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ab425-f9fe-448d-b4c3-d5282b2febfb admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:31.357640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ab425-f9fe-448d-b4c3-d5282b2febfb admin admin] PATH_INFO: `/v3/projects/6f2992c5de9b451384f54bcd036f05e0` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:31.358901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ab425-f9fe-448d-b4c3-d5282b2febfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2992c5de9b451384f54bcd036f05e0)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:31.361878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ab425-f9fe-448d-b4c3-d5282b2febfb admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:31.362882 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dded059-8b6e-4e60-9922-56cca3b1597b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:31.364839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4dded059-8b6e-4e60-9922-56cca3b1597b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:31.448642 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-225ab425-f9fe-448d-b4c3-d5282b2febfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2992c5de9b451384f54bcd036f05e0', 'request_id': 'req-225ab425-f9fe-448d-b4c3-d5282b2febfb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:31.456810 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-225ab425-f9fe-448d-b4c3-d5282b2febfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2992c5de9b451384f54bcd036f05e0', 'request_id': 'req-225ab425-f9fe-448d-b4c3-d5282b2febfb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:31.466586 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dded059-8b6e-4e60-9922-56cca3b1597b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:31.467418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-225ab425-f9fe-448d-b4c3-d5282b2febfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2992c5de9b451384f54bcd036f05e0', 'request_id': 'req-225ab425-f9fe-448d-b4c3-d5282b2febfb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:31.483234 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-225ab425-f9fe-448d-b4c3-d5282b2febfb admin admin] The token cache is being invalidate because project 6f2992c5de9b451384f54bcd036f05e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:31.483575 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-225ab425-f9fe-448d-b4c3-d5282b2febfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:31.484253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-225ab425-f9fe-448d-b4c3-d5282b2febfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:31.487634 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 989/2015] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:31 2022] DELETE /identity/v3/projects/6f2992c5de9b451384f54bcd036f05e0 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:31.492957 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d5674-6e58-4dcd-9241-96fea6836ec6 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:31.493550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d5674-6e58-4dcd-9241-96fea6836ec6 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:31.493932 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d5674-6e58-4dcd-9241-96fea6836ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:31.546059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-363d5674-6e58-4dcd-9241-96fea6836ec6 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:31.547463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-363d5674-6e58-4dcd-9241-96fea6836ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:31.548791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a649b-be0e-42b5-8c2e-62c5a185fa1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:31.549464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a649b-be0e-42b5-8c2e-62c5a185fa1a service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:31.549754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a649b-be0e-42b5-8c2e-62c5a185fa1a service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:31.550030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a649b-be0e-42b5-8c2e-62c5a185fa1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:31.552228 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-6e0a649b-be0e-42b5-8c2e-62c5a185fa1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:31.643390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 990/2016] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:31.651215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cccd82-c918-4568-a8e8-36cf2bad1992 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:31.652089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cccd82-c918-4568-a8e8-36cf2bad1992 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:31.652375 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cccd82-c918-4568-a8e8-36cf2bad1992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:31.678888 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-91cccd82-c918-4568-a8e8-36cf2bad1992 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:31.681277 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-91cccd82-c918-4568-a8e8-36cf2bad1992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:31.681411 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a649b-be0e-42b5-8c2e-62c5a185fa1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:31.682852 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a649b-be0e-42b5-8c2e-62c5a185fa1a service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:31.690924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 991/2017] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:31.695159 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cccd82-c918-4568-a8e8-36cf2bad1992 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:31.698089 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1027/2018] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:31.707785 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cccd82-c918-4568-a8e8-36cf2bad1992 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:31.714500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b54f3d-5a6c-4739-8965-9a5ec702be9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:31.716246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b54f3d-5a6c-4739-8965-9a5ec702be9c admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:31.717104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b54f3d-5a6c-4739-8965-9a5ec702be9c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:31.717564 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b54f3d-5a6c-4739-8965-9a5ec702be9c admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:31.718465 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b54f3d-5a6c-4739-8965-9a5ec702be9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:30:31.718959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b54f3d-5a6c-4739-8965-9a5ec702be9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:31.720781 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b54f3d-5a6c-4739-8965-9a5ec702be9c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:31.737062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 992/2019] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:30:31.741366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a649b-be0e-42b5-8c2e-62c5a185fa1a service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:31.754304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a649b-be0e-42b5-8c2e-62c5a185fa1a service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:31.769629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec88e7-42fc-4c30-8ab5-870da6fe2a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:31.772153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec88e7-42fc-4c30-8ab5-870da6fe2a34 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:31.772153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec88e7-42fc-4c30-8ab5-870da6fe2a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:31.772430 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec88e7-42fc-4c30-8ab5-870da6fe2a34 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:31.773145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec88e7-42fc-4c30-8ab5-870da6fe2a34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:31.777029 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec88e7-42fc-4c30-8ab5-870da6fe2a34 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:31.836429 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1028/2020] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:31 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:30:31.841755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b54f3d-5a6c-4739-8965-9a5ec702be9c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:31.855455 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b54f3d-5a6c-4739-8965-9a5ec702be9c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:32.045805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb168ab-92bd-4e81-88ba-d1cbf6e71902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:32.047199 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb168ab-92bd-4e81-88ba-d1cbf6e71902 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:32.047571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb168ab-92bd-4e81-88ba-d1cbf6e71902 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:32.047897 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb168ab-92bd-4e81-88ba-d1cbf6e71902 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:32.048690 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb168ab-92bd-4e81-88ba-d1cbf6e71902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:32.051934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb168ab-92bd-4e81-88ba-d1cbf6e71902 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:32.092053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 993/2021] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:31 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:30:32.095424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec88e7-42fc-4c30-8ab5-870da6fe2a34 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:32.111417 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec88e7-42fc-4c30-8ab5-870da6fe2a34 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:32.118124 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb00f70-8c84-443a-8fcc-439c7742bc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:32.119075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb00f70-8c84-443a-8fcc-439c7742bc86 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:32.119532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb00f70-8c84-443a-8fcc-439c7742bc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:32.119955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb00f70-8c84-443a-8fcc-439c7742bc86 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:32.120917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb00f70-8c84-443a-8fcc-439c7742bc86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:32.123602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb00f70-8c84-443a-8fcc-439c7742bc86 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:32.137961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1029/2022] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:32.142629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb168ab-92bd-4e81-88ba-d1cbf6e71902 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:32.156871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb168ab-92bd-4e81-88ba-d1cbf6e71902 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:32.162647 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14d8ae-f36c-46cf-a175-ae13602690f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:32.163749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14d8ae-f36c-46cf-a175-ae13602690f2 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:32.164207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14d8ae-f36c-46cf-a175-ae13602690f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:32.164620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14d8ae-f36c-46cf-a175-ae13602690f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:32.165551 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14d8ae-f36c-46cf-a175-ae13602690f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:32.167192 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14d8ae-f36c-46cf-a175-ae13602690f2 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:32.184188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 994/2023] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:32.189373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb00f70-8c84-443a-8fcc-439c7742bc86 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:32.203639 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb00f70-8c84-443a-8fcc-439c7742bc86 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:32.209560 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a64088-e164-49ef-b064-2afe74d9f66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:32.210576 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a64088-e164-49ef-b064-2afe74d9f66a admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:32.210923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a64088-e164-49ef-b064-2afe74d9f66a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:32.211324 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a64088-e164-49ef-b064-2afe74d9f66a admin admin] PATH_INFO: `/v3/projects/5ccc781135024a86a719aab2736034ef/users/b42135b59eb443b1964b1eb579315153/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:32.229916 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a64088-e164-49ef-b064-2afe74d9f66a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ccc781135024a86a719aab2736034ef, user_id=b42135b59eb443b1964b1eb579315153, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:32.235532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a64088-e164-49ef-b064-2afe74d9f66a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:32.259878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1030/2024] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:32 2022] PUT /identity/v3/projects/5ccc781135024a86a719aab2736034ef/users/b42135b59eb443b1964b1eb579315153/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:32.266344 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba9e1-568e-43f5-b645-9dc82cb569de None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:32.266639 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba9e1-568e-43f5-b645-9dc82cb569de None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:32.266951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba9e1-568e-43f5-b645-9dc82cb569de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:32.298642 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a64088-e164-49ef-b064-2afe74d9f66a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:32.300822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-50a64088-e164-49ef-b064-2afe74d9f66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:32.315527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-bb1ba9e1-568e-43f5-b645-9dc82cb569de None None] MFA Rules not processed for user `b42135b59eb443b1964b1eb579315153`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:32.317488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-bb1ba9e1-568e-43f5-b645-9dc82cb569de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:32.566890 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 995/2025] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:32.576888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a64088-e164-49ef-b064-2afe74d9f66a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:32.578646 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ba9e1-568e-43f5-b645-9dc82cb569de None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:32.581649 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-bb1ba9e1-568e-43f5-b645-9dc82cb569de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:32.614804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae28ef-cb64-4334-9506-e47ab6d2410c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:32.615736 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae28ef-cb64-4334-9506-e47ab6d2410c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:32.616022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae28ef-cb64-4334-9506-e47ab6d2410c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:32.616302 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae28ef-cb64-4334-9506-e47ab6d2410c admin admin] PATH_INFO: `/v3/projects/22b5c7e0342346b9baa3f4d8a77a6fef` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:32.626696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae28ef-cb64-4334-9506-e47ab6d2410c admin admin] RBAC: Authorizing `identity:delete_project(project_id=22b5c7e0342346b9baa3f4d8a77a6fef)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:32.630603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae28ef-cb64-4334-9506-e47ab6d2410c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:32.807757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ba9e1-568e-43f5-b645-9dc82cb569de None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:32.821359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-eeae28ef-cb64-4334-9506-e47ab6d2410c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22b5c7e0342346b9baa3f4d8a77a6fef', 'request_id': 'req-eeae28ef-cb64-4334-9506-e47ab6d2410c'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:32.861894 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-eeae28ef-cb64-4334-9506-e47ab6d2410c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22b5c7e0342346b9baa3f4d8a77a6fef', 'request_id': 'req-eeae28ef-cb64-4334-9506-e47ab6d2410c'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:32.891372 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-eeae28ef-cb64-4334-9506-e47ab6d2410c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22b5c7e0342346b9baa3f4d8a77a6fef', 'request_id': 'req-eeae28ef-cb64-4334-9506-e47ab6d2410c'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:32.939201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-eeae28ef-cb64-4334-9506-e47ab6d2410c admin admin] The token cache is being invalidate because project 22b5c7e0342346b9baa3f4d8a77a6fef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:32.939516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-eeae28ef-cb64-4334-9506-e47ab6d2410c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:32.948284 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-eeae28ef-cb64-4334-9506-e47ab6d2410c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:32.951058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1031/2026] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:32 2022] DELETE /identity/v3/projects/22b5c7e0342346b9baa3f4d8a77a6fef => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:32.965883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a56a78-508a-49f8-a937-71ed00c2dc23 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:32.966209 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a56a78-508a-49f8-a937-71ed00c2dc23 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:32.966582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a56a78-508a-49f8-a937-71ed00c2dc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:33.090161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-82a56a78-508a-49f8-a937-71ed00c2dc23 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:33.093706 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-82a56a78-508a-49f8-a937-71ed00c2dc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:33.111235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b3570-f4ab-42dc-87a9-841a0feb8c42 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cf284e81ba4042b66c0bc189f1ae2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:33.111779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b3570-f4ab-42dc-87a9-841a0feb8c42 service glance] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:33.111779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b3570-f4ab-42dc-87a9-841a0feb8c42 service glance] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:33.111920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b3570-f4ab-42dc-87a9-841a0feb8c42 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:33.120908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7e5b3570-f4ab-42dc-87a9-841a0feb8c42 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:33.256673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1032/2027] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:33.263459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd0b63-0b8a-410f-a463-c52767188405 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:33.263697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd0b63-0b8a-410f-a463-c52767188405 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:33.263997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd0b63-0b8a-410f-a463-c52767188405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:33.295788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-00dd0b63-0b8a-410f-a463-c52767188405 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:33.299461 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-00dd0b63-0b8a-410f-a463-c52767188405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:33.310669 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1033/2028] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:33.311626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd0b63-0b8a-410f-a463-c52767188405 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:33.320893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd0b63-0b8a-410f-a463-c52767188405 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:33.327344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895959f-8864-4a18-9b5a-407d7300faac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:33.327344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895959f-8864-4a18-9b5a-407d7300faac admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:33.327751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895959f-8864-4a18-9b5a-407d7300faac admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:33.328030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895959f-8864-4a18-9b5a-407d7300faac admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:33.328954 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895959f-8864-4a18-9b5a-407d7300faac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:30:33.329266 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895959f-8864-4a18-9b5a-407d7300faac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:33.333108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895959f-8864-4a18-9b5a-407d7300faac admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:33.350169 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b3570-f4ab-42dc-87a9-841a0feb8c42 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:33.353437 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b3570-f4ab-42dc-87a9-841a0feb8c42 service glance] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:33.354755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1034/2029] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:30:33.363339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895959f-8864-4a18-9b5a-407d7300faac admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:33.375278 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895959f-8864-4a18-9b5a-407d7300faac admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:33.395840 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 996/2030] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 817 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:30:33.399317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c0208-7d40-4647-8732-9343d0dad7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:33.400079 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c0208-7d40-4647-8732-9343d0dad7fc admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:33.400410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c0208-7d40-4647-8732-9343d0dad7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:33.400693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c0208-7d40-4647-8732-9343d0dad7fc admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:33.403208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c0208-7d40-4647-8732-9343d0dad7fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:33.406609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c0208-7d40-4647-8732-9343d0dad7fc admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:33.465858 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1035/2031] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:33 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:33.470055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b3570-f4ab-42dc-87a9-841a0feb8c42 service glance] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:33.510347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b3570-f4ab-42dc-87a9-841a0feb8c42 service glance] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:33.639938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2f0cc-a429-4d9f-9d17-0c59aface932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:33.644916 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2f0cc-a429-4d9f-9d17-0c59aface932 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:33.645267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2f0cc-a429-4d9f-9d17-0c59aface932 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:33.645555 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2f0cc-a429-4d9f-9d17-0c59aface932 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:33.646417 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2f0cc-a429-4d9f-9d17-0c59aface932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:33.649995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2f0cc-a429-4d9f-9d17-0c59aface932 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:33.685889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 997/2032] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:33 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:33.690230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c0208-7d40-4647-8732-9343d0dad7fc admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:33.710866 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c0208-7d40-4647-8732-9343d0dad7fc admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:33.723384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0ba5f-98b9-4a61-b02a-8af8eeea13d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:33.724073 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0ba5f-98b9-4a61-b02a-8af8eeea13d2 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:33.724358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0ba5f-98b9-4a61-b02a-8af8eeea13d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:33.724635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0ba5f-98b9-4a61-b02a-8af8eeea13d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:33.725380 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0ba5f-98b9-4a61-b02a-8af8eeea13d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:33.733682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0ba5f-98b9-4a61-b02a-8af8eeea13d2 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:33.754410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1036/2033] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:33.758446 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2f0cc-a429-4d9f-9d17-0c59aface932 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:33.769676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2f0cc-a429-4d9f-9d17-0c59aface932 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:33.776852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721b4c0-9475-4f24-b0f9-fe45fecf2efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:33.779137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721b4c0-9475-4f24-b0f9-fe45fecf2efd admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:33.780249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721b4c0-9475-4f24-b0f9-fe45fecf2efd admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:33.781213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721b4c0-9475-4f24-b0f9-fe45fecf2efd admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:33.783455 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721b4c0-9475-4f24-b0f9-fe45fecf2efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:33.785831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721b4c0-9475-4f24-b0f9-fe45fecf2efd admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:33.807395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 998/2034] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:33.808607 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0ba5f-98b9-4a61-b02a-8af8eeea13d2 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:33.820465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0ba5f-98b9-4a61-b02a-8af8eeea13d2 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:33.839139 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa01a7-ddfc-42ec-aabc-063ae0451762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:33.840051 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa01a7-ddfc-42ec-aabc-063ae0451762 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:33.840330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa01a7-ddfc-42ec-aabc-063ae0451762 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:33.840612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa01a7-ddfc-42ec-aabc-063ae0451762 admin admin] PATH_INFO: `/v3/projects/9c6e0a23a3414fca84248553e4f85005/users/96dcd82937834f89889c43a1fefc1bff/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:33.869011 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa01a7-ddfc-42ec-aabc-063ae0451762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c6e0a23a3414fca84248553e4f85005, user_id=96dcd82937834f89889c43a1fefc1bff, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:33.872171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa01a7-ddfc-42ec-aabc-063ae0451762 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:33.900943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1037/2035] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:33 2022] PUT /identity/v3/projects/9c6e0a23a3414fca84248553e4f85005/users/96dcd82937834f89889c43a1fefc1bff/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:33.904553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c47a1-ab53-4b42-bf60-e2c4b6535965 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:33.904553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c47a1-ab53-4b42-bf60-e2c4b6535965 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:33.904754 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c47a1-ab53-4b42-bf60-e2c4b6535965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:33.951111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-b63c47a1-ab53-4b42-bf60-e2c4b6535965 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:33.956951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b63c47a1-ab53-4b42-bf60-e2c4b6535965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:34.035710 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 999/2036] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:34.046763 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa01a7-ddfc-42ec-aabc-063ae0451762 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:34.050364 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7dfa01a7-ddfc-42ec-aabc-063ae0451762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:34.177823 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa01a7-ddfc-42ec-aabc-063ae0451762 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:34.193509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f7243-da5b-4d45-8e13-91679d6420bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:34.194181 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f7243-da5b-4d45-8e13-91679d6420bf service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:34.194464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f7243-da5b-4d45-8e13-91679d6420bf service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:34.194740 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f7243-da5b-4d45-8e13-91679d6420bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:34.250126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f7243-da5b-4d45-8e13-91679d6420bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:34.253621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f7243-da5b-4d45-8e13-91679d6420bf service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:34.270913 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1038/2037] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:35.871809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c47a1-ab53-4b42-bf60-e2c4b6535965 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:35.873252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b63c47a1-ab53-4b42-bf60-e2c4b6535965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:35.985904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c47a1-ab53-4b42-bf60-e2c4b6535965 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:35.989998 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2d97f-df11-402f-80d4-dbbc83257355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:35.990686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2d97f-df11-402f-80d4-dbbc83257355 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:35.990971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2d97f-df11-402f-80d4-dbbc83257355 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:35.991576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2d97f-df11-402f-80d4-dbbc83257355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:35.995410 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-75c2d97f-df11-402f-80d4-dbbc83257355 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:36.116765 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2d97f-df11-402f-80d4-dbbc83257355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:36.118684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2d97f-df11-402f-80d4-dbbc83257355 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:36.134048 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1000/2038] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:41.190317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3306233-a804-41bd-b1f6-4ae78b6ef287 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:41.191961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3306233-a804-41bd-b1f6-4ae78b6ef287 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:41.192645 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3306233-a804-41bd-b1f6-4ae78b6ef287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:41.268805 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-d3306233-a804-41bd-b1f6-4ae78b6ef287 None None] MFA Rules not processed for user `96dcd82937834f89889c43a1fefc1bff`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:41.271048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-d3306233-a804-41bd-b1f6-4ae78b6ef287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:41.347396 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1039/2039] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:41.351886 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2d97f-df11-402f-80d4-dbbc83257355 service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:41.366384 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2d97f-df11-402f-80d4-dbbc83257355 service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:41.372985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9452e-27bb-49ce-8494-fa3abba80853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:41.374325 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9452e-27bb-49ce-8494-fa3abba80853 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:41.375294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9452e-27bb-49ce-8494-fa3abba80853 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:41.375742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9452e-27bb-49ce-8494-fa3abba80853 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:41.377030 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9452e-27bb-49ce-8494-fa3abba80853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:41.379362 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9452e-27bb-49ce-8494-fa3abba80853 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:41.415971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1001/2040] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:41 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:41.420526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3306233-a804-41bd-b1f6-4ae78b6ef287 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:41.435711 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3306233-a804-41bd-b1f6-4ae78b6ef287 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:41.527749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c3e9c-1b1d-4388-919b-95e508d94632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:41.529073 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c3e9c-1b1d-4388-919b-95e508d94632 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:41.529374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c3e9c-1b1d-4388-919b-95e508d94632 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:41.529700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c3e9c-1b1d-4388-919b-95e508d94632 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:41.530420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c3e9c-1b1d-4388-919b-95e508d94632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:41.532543 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c3e9c-1b1d-4388-919b-95e508d94632 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:41.562504 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1040/2041] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:41 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:41.566543 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9452e-27bb-49ce-8494-fa3abba80853 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:41.582019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9452e-27bb-49ce-8494-fa3abba80853 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:41.590108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b20f8-eca7-41a8-81ca-b71586822459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:41.591767 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b20f8-eca7-41a8-81ca-b71586822459 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:41.592264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b20f8-eca7-41a8-81ca-b71586822459 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:41.592868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b20f8-eca7-41a8-81ca-b71586822459 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:41.593992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b20f8-eca7-41a8-81ca-b71586822459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:41.596074 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b20f8-eca7-41a8-81ca-b71586822459 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:41.610531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1002/2042] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:30:41.614968 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c3e9c-1b1d-4388-919b-95e508d94632 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:41.627457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c3e9c-1b1d-4388-919b-95e508d94632 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:41.633041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab83b7-0e28-401b-905e-110cfd473e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:41.634022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab83b7-0e28-401b-905e-110cfd473e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:41.634501 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab83b7-0e28-401b-905e-110cfd473e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:41.634792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab83b7-0e28-401b-905e-110cfd473e4c admin admin] PATH_INFO: `/v3/projects/62d7aa11427b4370a7e450a99ca61594/users/1be77b4735634f718f55bc6de7fd28c5/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:41.660605 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab83b7-0e28-401b-905e-110cfd473e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d7aa11427b4370a7e450a99ca61594, user_id=1be77b4735634f718f55bc6de7fd28c5, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:41.664075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab83b7-0e28-401b-905e-110cfd473e4c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:41.690134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1041/2043] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:41 2022] PUT /identity/v3/projects/62d7aa11427b4370a7e450a99ca61594/users/1be77b4735634f718f55bc6de7fd28c5/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:41.694019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b20f8-eca7-41a8-81ca-b71586822459 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:41.711095 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b20f8-eca7-41a8-81ca-b71586822459 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:41.800466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70daf709-2001-4bc2-b043-956e1443f0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:41.801264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daf709-2001-4bc2-b043-956e1443f0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:41.801704 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daf709-2001-4bc2-b043-956e1443f0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:41.801994 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daf709-2001-4bc2-b043-956e1443f0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:41.802735 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70daf709-2001-4bc2-b043-956e1443f0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:41.804389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70daf709-2001-4bc2-b043-956e1443f0c3 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:41.821610 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1003/2044] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:41.826156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab83b7-0e28-401b-905e-110cfd473e4c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:41.841247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab83b7-0e28-401b-905e-110cfd473e4c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:41.851819 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94b58b-0354-495d-92ed-84eb402c7056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:41.858124 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94b58b-0354-495d-92ed-84eb402c7056 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:41.858434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94b58b-0354-495d-92ed-84eb402c7056 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:41.858718 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94b58b-0354-495d-92ed-84eb402c7056 admin admin] PATH_INFO: `/v3/projects/62d7aa11427b4370a7e450a99ca61594/users/1be77b4735634f718f55bc6de7fd28c5/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:41.861388 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94b58b-0354-495d-92ed-84eb402c7056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d7aa11427b4370a7e450a99ca61594, user_id=1be77b4735634f718f55bc6de7fd28c5, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:41.864050 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94b58b-0354-495d-92ed-84eb402c7056 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:41.878612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1042/2045] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:41 2022] PUT /identity/v3/projects/62d7aa11427b4370a7e450a99ca61594/users/1be77b4735634f718f55bc6de7fd28c5/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:41.956754 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70daf709-2001-4bc2-b043-956e1443f0c3 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:41.958669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-70daf709-2001-4bc2-b043-956e1443f0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:41.980484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70daf709-2001-4bc2-b043-956e1443f0c3 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:42.046430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094b2a3-f869-4a52-ba60-ac470e0629f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:42.047832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094b2a3-f869-4a52-ba60-ac470e0629f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:42.048124 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094b2a3-f869-4a52-ba60-ac470e0629f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:42.048443 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094b2a3-f869-4a52-ba60-ac470e0629f2 admin admin] PATH_INFO: `/v3/users/75b86c0021e7411b9e4f7cc1e65b4f2b` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:42.049908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094b2a3-f869-4a52-ba60-ac470e0629f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75b86c0021e7411b9e4f7cc1e65b4f2b)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:42.051869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094b2a3-f869-4a52-ba60-ac470e0629f2 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:42.103489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d094b2a3-f869-4a52-ba60-ac470e0629f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75b86c0021e7411b9e4f7cc1e65b4f2b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:42.111215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d094b2a3-f869-4a52-ba60-ac470e0629f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75b86c0021e7411b9e4f7cc1e65b4f2b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:42.112081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d094b2a3-f869-4a52-ba60-ac470e0629f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75b86c0021e7411b9e4f7cc1e65b4f2b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:42.120687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d094b2a3-f869-4a52-ba60-ac470e0629f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75b86c0021e7411b9e4f7cc1e65b4f2b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:42.122903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d094b2a3-f869-4a52-ba60-ac470e0629f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75b86c0021e7411b9e4f7cc1e65b4f2b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:42.143168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1004/2046] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:41 2022] DELETE /identity/v3/users/75b86c0021e7411b9e4f7cc1e65b4f2b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:42.147777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df0913-fdae-4adb-a1a3-57d1911fbfe5 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:42.148066 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df0913-fdae-4adb-a1a3-57d1911fbfe5 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:42.148345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df0913-fdae-4adb-a1a3-57d1911fbfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:42.165543 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-06df0913-fdae-4adb-a1a3-57d1911fbfe5 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:42.166929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-06df0913-fdae-4adb-a1a3-57d1911fbfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:42.252207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1043/2047] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:42.258799 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094b2a3-f869-4a52-ba60-ac470e0629f2 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:42.259218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-d094b2a3-f869-4a52-ba60-ac470e0629f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:42.377742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094b2a3-f869-4a52-ba60-ac470e0629f2 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:42.381522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0451460-af79-4b12-9f63-a5eca8103752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:42.382193 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0451460-af79-4b12-9f63-a5eca8103752 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:42.382532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0451460-af79-4b12-9f63-a5eca8103752 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:42.382817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0451460-af79-4b12-9f63-a5eca8103752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:42.395837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0451460-af79-4b12-9f63-a5eca8103752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:42.398205 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0451460-af79-4b12-9f63-a5eca8103752 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:42.412865 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1005/2048] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:43.027154 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df0913-fdae-4adb-a1a3-57d1911fbfe5 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:43.029620 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-06df0913-fdae-4adb-a1a3-57d1911fbfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:43.048984 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df0913-fdae-4adb-a1a3-57d1911fbfe5 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:43.059615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:43.061046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:43.063082 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:43.063523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] PATH_INFO: `/v3/projects/63886a179df746d4b2a750f8054028cf` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:43.065193 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=63886a179df746d4b2a750f8054028cf)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:43.067369 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:43.150258 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63886a179df746d4b2a750f8054028cf', 'request_id': 'req-3ce20680-4c11-4a51-8110-482bf0890c7b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:43.158094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63886a179df746d4b2a750f8054028cf', 'request_id': 'req-3ce20680-4c11-4a51-8110-482bf0890c7b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:43.172223 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63886a179df746d4b2a750f8054028cf', 'request_id': 'req-3ce20680-4c11-4a51-8110-482bf0890c7b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:43.186433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] The token cache is being invalidate because project 63886a179df746d4b2a750f8054028cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:43.186822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:43.187819 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:43.189952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1044/2049] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:43 2022] DELETE /identity/v3/projects/63886a179df746d4b2a750f8054028cf => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:43.194168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0451460-af79-4b12-9f63-a5eca8103752 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:43.212606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0451460-af79-4b12-9f63-a5eca8103752 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:43.337849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c88f5f-673f-496d-bae1-0e621478fb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:43.338849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88f5f-673f-496d-bae1-0e621478fb47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:43.339140 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88f5f-673f-496d-bae1-0e621478fb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:43.339486 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88f5f-673f-496d-bae1-0e621478fb47 admin admin] PATH_INFO: `/v3/users/a733d683dbd347648aacb5062a755f0c` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:43.343268 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c88f5f-673f-496d-bae1-0e621478fb47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a733d683dbd347648aacb5062a755f0c)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:43.344826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c88f5f-673f-496d-bae1-0e621478fb47 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:43.410946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-83c88f5f-673f-496d-bae1-0e621478fb47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a733d683dbd347648aacb5062a755f0c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:43.418392 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-83c88f5f-673f-496d-bae1-0e621478fb47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a733d683dbd347648aacb5062a755f0c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:43.420374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-83c88f5f-673f-496d-bae1-0e621478fb47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a733d683dbd347648aacb5062a755f0c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:43.437563 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-83c88f5f-673f-496d-bae1-0e621478fb47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a733d683dbd347648aacb5062a755f0c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:43.438741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-83c88f5f-673f-496d-bae1-0e621478fb47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a733d683dbd347648aacb5062a755f0c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:43.469500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1006/2050] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:43 2022] DELETE /identity/v3/users/a733d683dbd347648aacb5062a755f0c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:43.840940 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:43.843215 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:43.873599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce20680-4c11-4a51-8110-482bf0890c7b admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:43.997621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2868832-4047-4292-8849-e9f089d946e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:43.997621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2868832-4047-4292-8849-e9f089d946e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:43.997621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2868832-4047-4292-8849-e9f089d946e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:43.997621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2868832-4047-4292-8849-e9f089d946e1 admin admin] PATH_INFO: `/v3/users/3312802ff729418f90e6f25e6faf0ee0` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:43.998073 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2868832-4047-4292-8849-e9f089d946e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3312802ff729418f90e6f25e6faf0ee0)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:43.999664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2868832-4047-4292-8849-e9f089d946e1 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:44.018780 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c88f5f-673f-496d-bae1-0e621478fb47 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:44.020310 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-83c88f5f-673f-496d-bae1-0e621478fb47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:44.048186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c88f5f-673f-496d-bae1-0e621478fb47 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:44.064689 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:44.068031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:44.068444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:44.070521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21 admin admin] PATH_INFO: `/v3/projects/542388eeb32d4227bd4ae743d7e9dc14` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:44.073995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=542388eeb32d4227bd4ae743d7e9dc14)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:44.078224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:44.088914 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e2868832-4047-4292-8849-e9f089d946e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3312802ff729418f90e6f25e6faf0ee0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.088914 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e2868832-4047-4292-8849-e9f089d946e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3312802ff729418f90e6f25e6faf0ee0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.101990 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e2868832-4047-4292-8849-e9f089d946e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3312802ff729418f90e6f25e6faf0ee0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.115443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e2868832-4047-4292-8849-e9f089d946e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3312802ff729418f90e6f25e6faf0ee0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.116656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e2868832-4047-4292-8849-e9f089d946e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3312802ff729418f90e6f25e6faf0ee0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.146816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '542388eeb32d4227bd4ae743d7e9dc14', 'request_id': 'req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.153033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '542388eeb32d4227bd4ae743d7e9dc14', 'request_id': 'req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.156483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1045/2051] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:43 2022] DELETE /identity/v3/users/3312802ff729418f90e6f25e6faf0ee0 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:44.170711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '542388eeb32d4227bd4ae743d7e9dc14', 'request_id': 'req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.191506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21 admin admin] The token cache is being invalidate because project 542388eeb32d4227bd4ae743d7e9dc14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:44.192397 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.193924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-463a3e77-5119-4ebb-a25a-eb3fb4cc7b21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.201748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1007/2052] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:44 2022] DELETE /identity/v3/projects/542388eeb32d4227bd4ae743d7e9dc14 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:44.207724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db39452-9a19-44ea-abc3-ee689b9ac955 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:44.208217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db39452-9a19-44ea-abc3-ee689b9ac955 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:44.208501 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db39452-9a19-44ea-abc3-ee689b9ac955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:44.229699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-7db39452-9a19-44ea-abc3-ee689b9ac955 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:44.230960 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7db39452-9a19-44ea-abc3-ee689b9ac955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:44.323542 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1046/2053] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:44.328176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1635373-7513-4b13-afe0-32a6a5313813 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:44.328819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1635373-7513-4b13-afe0-32a6a5313813 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:44.329138 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1635373-7513-4b13-afe0-32a6a5313813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:44.371058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f1635373-7513-4b13-afe0-32a6a5313813 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:44.373910 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f1635373-7513-4b13-afe0-32a6a5313813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:44.394321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1008/2054] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:44.399408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db39452-9a19-44ea-abc3-ee689b9ac955 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:44.414623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1635373-7513-4b13-afe0-32a6a5313813 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:44.416745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f1635373-7513-4b13-afe0-32a6a5313813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:44.417554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db39452-9a19-44ea-abc3-ee689b9ac955 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:44.430502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29be5c2-fe93-4e48-83da-f9f3143d074a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:44.431217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29be5c2-fe93-4e48-83da-f9f3143d074a admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:44.431765 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29be5c2-fe93-4e48-83da-f9f3143d074a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:44.431985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29be5c2-fe93-4e48-83da-f9f3143d074a admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:44.432551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29be5c2-fe93-4e48-83da-f9f3143d074a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:30:44.432864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29be5c2-fe93-4e48-83da-f9f3143d074a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:44.435008 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29be5c2-fe93-4e48-83da-f9f3143d074a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:44.448619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1047/2055] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:30:44.454637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1635373-7513-4b13-afe0-32a6a5313813 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:44.455005 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29be5c2-fe93-4e48-83da-f9f3143d074a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:44.478640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29be5c2-fe93-4e48-83da-f9f3143d074a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:44.482126 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4c1ef-0c7a-430c-894a-87943f05b36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:44.483155 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4c1ef-0c7a-430c-894a-87943f05b36b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:44.483543 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4c1ef-0c7a-430c-894a-87943f05b36b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:44.483951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4c1ef-0c7a-430c-894a-87943f05b36b admin admin] PATH_INFO: `/v3/projects/3e0d940b1aea46aea0fa187448375f04` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:44.488027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4c1ef-0c7a-430c-894a-87943f05b36b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e0d940b1aea46aea0fa187448375f04)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:44.497385 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87dee48-6fb1-4fe9-b8c0-e800025d043c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:44.498602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87dee48-6fb1-4fe9-b8c0-e800025d043c admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:44.499056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87dee48-6fb1-4fe9-b8c0-e800025d043c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:44.499524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87dee48-6fb1-4fe9-b8c0-e800025d043c admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:44.500397 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4c1ef-0c7a-430c-894a-87943f05b36b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:44.500814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87dee48-6fb1-4fe9-b8c0-e800025d043c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:44.503458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87dee48-6fb1-4fe9-b8c0-e800025d043c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:44.577897 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1048/2056] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:44 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:44.582312 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87dee48-6fb1-4fe9-b8c0-e800025d043c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:44.601171 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-17b4c1ef-0c7a-430c-894a-87943f05b36b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e0d940b1aea46aea0fa187448375f04', 'request_id': 'req-17b4c1ef-0c7a-430c-894a-87943f05b36b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.615776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-17b4c1ef-0c7a-430c-894a-87943f05b36b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e0d940b1aea46aea0fa187448375f04', 'request_id': 'req-17b4c1ef-0c7a-430c-894a-87943f05b36b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.630133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-17b4c1ef-0c7a-430c-894a-87943f05b36b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e0d940b1aea46aea0fa187448375f04', 'request_id': 'req-17b4c1ef-0c7a-430c-894a-87943f05b36b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.651123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-17b4c1ef-0c7a-430c-894a-87943f05b36b admin admin] The token cache is being invalidate because project 3e0d940b1aea46aea0fa187448375f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:30:44.651707 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-17b4c1ef-0c7a-430c-894a-87943f05b36b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.652571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-17b4c1ef-0c7a-430c-894a-87943f05b36b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:30:44.654844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1009/2057] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:30:44 2022] DELETE /identity/v3/projects/3e0d940b1aea46aea0fa187448375f04 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:44.659887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee13c27-d72e-4349-9db2-ff18b136c865 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:44.660385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee13c27-d72e-4349-9db2-ff18b136c865 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:44.661086 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee13c27-d72e-4349-9db2-ff18b136c865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:44.705035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-3ee13c27-d72e-4349-9db2-ff18b136c865 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:44.706695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-3ee13c27-d72e-4349-9db2-ff18b136c865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:44.752353 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87dee48-6fb1-4fe9-b8c0-e800025d043c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:44.772683 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f3ab9-179e-4bb7-9df5-efdb15348284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:44.776530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f3ab9-179e-4bb7-9df5-efdb15348284 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:44.777077 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f3ab9-179e-4bb7-9df5-efdb15348284 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:44.777360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f3ab9-179e-4bb7-9df5-efdb15348284 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:44.778681 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f3ab9-179e-4bb7-9df5-efdb15348284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:44.780736 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f3ab9-179e-4bb7-9df5-efdb15348284 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:44.850211 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1049/2058] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:44 2022] POST /identity/v3/users => generated 292 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:44.853803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f3ab9-179e-4bb7-9df5-efdb15348284 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:44.860388 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1010/2059] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:44.864353 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-803f3ab9-179e-4bb7-9df5-efdb15348284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:44.868355 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cb696-49ab-43f1-9a94-336f5cd4c912 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:44.868758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cb696-49ab-43f1-9a94-336f5cd4c912 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:44.869133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cb696-49ab-43f1-9a94-336f5cd4c912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:44.905203 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f3ab9-179e-4bb7-9df5-efdb15348284 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:44.906945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f90cb696-49ab-43f1-9a94-336f5cd4c912 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:44.911846 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f90cb696-49ab-43f1-9a94-336f5cd4c912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:44.916786 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2d220-91f1-40a4-877e-1090499e7e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:44.917516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2d220-91f1-40a4-877e-1090499e7e16 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:44.917714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2d220-91f1-40a4-877e-1090499e7e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:44.917990 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2d220-91f1-40a4-877e-1090499e7e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:44.918757 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2d220-91f1-40a4-877e-1090499e7e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:44.922159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2d220-91f1-40a4-877e-1090499e7e16 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:44.932181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1011/2060] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:44.936617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cb696-49ab-43f1-9a94-336f5cd4c912 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:44.952610 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1050/2061] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:44.952828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cb696-49ab-43f1-9a94-336f5cd4c912 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:44.954743 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2d220-91f1-40a4-877e-1090499e7e16 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:44.966376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3cf53-66fd-4969-ac91-f7470d4eeecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:44.967545 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3cf53-66fd-4969-ac91-f7470d4eeecd admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:44.969131 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3cf53-66fd-4969-ac91-f7470d4eeecd admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:44.969500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3cf53-66fd-4969-ac91-f7470d4eeecd admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:44.970501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3cf53-66fd-4969-ac91-f7470d4eeecd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:30:44.972910 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3cf53-66fd-4969-ac91-f7470d4eeecd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:44.974643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3cf53-66fd-4969-ac91-f7470d4eeecd admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:44.975646 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2d220-91f1-40a4-877e-1090499e7e16 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:44.989424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d92f0-20d6-4f9e-878f-261299d6eecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:44.990214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d92f0-20d6-4f9e-878f-261299d6eecc admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:44.990565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d92f0-20d6-4f9e-878f-261299d6eecc admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:44.991958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d92f0-20d6-4f9e-878f-261299d6eecc admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:44.992676 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d92f0-20d6-4f9e-878f-261299d6eecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:44.999234 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d92f0-20d6-4f9e-878f-261299d6eecc admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:45.021269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1012/2062] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:30:45.022216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3cf53-66fd-4969-ac91-f7470d4eeecd admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:45.049400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1051/2063] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:45.053614 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d92f0-20d6-4f9e-878f-261299d6eecc admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:45.060753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3cf53-66fd-4969-ac91-f7470d4eeecd admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:45.083807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380acf2-2b6b-453f-a126-46afbdb9aae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:45.083955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d92f0-20d6-4f9e-878f-261299d6eecc admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:45.086288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380acf2-2b6b-453f-a126-46afbdb9aae6 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:45.086578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380acf2-2b6b-453f-a126-46afbdb9aae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:45.089048 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380acf2-2b6b-453f-a126-46afbdb9aae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:45.089850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380acf2-2b6b-453f-a126-46afbdb9aae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:45.091931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380acf2-2b6b-453f-a126-46afbdb9aae6 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:45.106748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc705b0d-1dbf-4a02-accf-2b003da3608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:45.108205 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc705b0d-1dbf-4a02-accf-2b003da3608f admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:45.108494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc705b0d-1dbf-4a02-accf-2b003da3608f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:45.108762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc705b0d-1dbf-4a02-accf-2b003da3608f admin admin] PATH_INFO: `/v3/projects/c31fb870496946e18fa3e115da89ae62/users/820df11e937c4c759286828443046e52/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:45.158376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc705b0d-1dbf-4a02-accf-2b003da3608f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31fb870496946e18fa3e115da89ae62, user_id=820df11e937c4c759286828443046e52, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:45.162553 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc705b0d-1dbf-4a02-accf-2b003da3608f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:45.166981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1013/2064] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:45 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:45.171126 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380acf2-2b6b-453f-a126-46afbdb9aae6 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:45.201732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1052/2065] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:45 2022] PUT /identity/v3/projects/c31fb870496946e18fa3e115da89ae62/users/820df11e937c4c759286828443046e52/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:45.204554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d161a-3f37-416b-aa27-6ec4a2626514 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:45.204859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d161a-3f37-416b-aa27-6ec4a2626514 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:45.205263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d161a-3f37-416b-aa27-6ec4a2626514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:45.292514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2b1d161a-3f37-416b-aa27-6ec4a2626514 None None] MFA Rules not processed for user `820df11e937c4c759286828443046e52`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:45.298142 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2b1d161a-3f37-416b-aa27-6ec4a2626514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:45.335893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380acf2-2b6b-453f-a126-46afbdb9aae6 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:45.448937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1053/2066] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:45.454455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d161a-3f37-416b-aa27-6ec4a2626514 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:45.468726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422a93d-2a88-4db9-8509-fa91a3727537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:45.470911 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422a93d-2a88-4db9-8509-fa91a3727537 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:45.471421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422a93d-2a88-4db9-8509-fa91a3727537 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:45.472000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422a93d-2a88-4db9-8509-fa91a3727537 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:45.473137 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422a93d-2a88-4db9-8509-fa91a3727537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:45.475845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422a93d-2a88-4db9-8509-fa91a3727537 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:45.507207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1014/2067] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:45 2022] POST /identity/v3/users => generated 293 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:30:45.511131 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422a93d-2a88-4db9-8509-fa91a3727537 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:45.524876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422a93d-2a88-4db9-8509-fa91a3727537 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:45.537889 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53916953-c1e0-4ea0-946c-d6a51ba663b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:45.538186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53916953-c1e0-4ea0-946c-d6a51ba663b5 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:45.538462 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53916953-c1e0-4ea0-946c-d6a51ba663b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:45.539452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53916953-c1e0-4ea0-946c-d6a51ba663b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:45.540228 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53916953-c1e0-4ea0-946c-d6a51ba663b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:45.542769 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53916953-c1e0-4ea0-946c-d6a51ba663b5 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:45.554042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d161a-3f37-416b-aa27-6ec4a2626514 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:45.560427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd833f9-c830-4df1-9feb-6e042414a55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:45.561108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd833f9-c830-4df1-9feb-6e042414a55a admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:45.561405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd833f9-c830-4df1-9feb-6e042414a55a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:45.561721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd833f9-c830-4df1-9feb-6e042414a55a admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:45.562797 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd833f9-c830-4df1-9feb-6e042414a55a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:45.562972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1015/2068] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:45.566853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd833f9-c830-4df1-9feb-6e042414a55a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:45.570527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53916953-c1e0-4ea0-946c-d6a51ba663b5 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:45.586524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53916953-c1e0-4ea0-946c-d6a51ba663b5 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:45.593775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f9114-05d1-470d-9b81-ed252d4a1f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:45.594524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f9114-05d1-470d-9b81-ed252d4a1f17 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:45.594818 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f9114-05d1-470d-9b81-ed252d4a1f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:45.595100 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f9114-05d1-470d-9b81-ed252d4a1f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:45.596050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f9114-05d1-470d-9b81-ed252d4a1f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:45.598098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f9114-05d1-470d-9b81-ed252d4a1f17 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:45.615783 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1016/2069] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:45.619572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f9114-05d1-470d-9b81-ed252d4a1f17 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:45.620546 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1054/2070] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:45 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:45.621577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd833f9-c830-4df1-9feb-6e042414a55a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:45.638786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f9114-05d1-470d-9b81-ed252d4a1f17 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:45.718724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd833f9-c830-4df1-9feb-6e042414a55a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:45.724333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c2322-c9f7-49f2-93b4-2276a2c1dcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:45.725463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c2322-c9f7-49f2-93b4-2276a2c1dcb0 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:45.725891 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c2322-c9f7-49f2-93b4-2276a2c1dcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:45.726347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c2322-c9f7-49f2-93b4-2276a2c1dcb0 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:45.729609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c2322-c9f7-49f2-93b4-2276a2c1dcb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:45.731977 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c2322-c9f7-49f2-93b4-2276a2c1dcb0 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:45.773661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1055/2071] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:45 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:45.778079 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c2322-c9f7-49f2-93b4-2276a2c1dcb0 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:45.788688 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c2322-c9f7-49f2-93b4-2276a2c1dcb0 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:45.790448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf880c-409a-4492-afa8-4471c7967734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:45.791480 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf880c-409a-4492-afa8-4471c7967734 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:45.793952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb6f01-da2d-4dc5-a5a0-1e0c0bea1548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:45.794512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf880c-409a-4492-afa8-4471c7967734 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:45.795035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf880c-409a-4492-afa8-4471c7967734 admin admin] PATH_INFO: `/v3/projects/127c608e5ebd478282b045739182c41e/users/cac32da403af402c8236f46168e81038/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:45.795339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6f01-da2d-4dc5-a5a0-1e0c0bea1548 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:45.795774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6f01-da2d-4dc5-a5a0-1e0c0bea1548 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:45.796185 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6f01-da2d-4dc5-a5a0-1e0c0bea1548 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:45.797064 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb6f01-da2d-4dc5-a5a0-1e0c0bea1548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:45.799155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb6f01-da2d-4dc5-a5a0-1e0c0bea1548 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:45.818971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1056/2072] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:45.820179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb6f01-da2d-4dc5-a5a0-1e0c0bea1548 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:45.832231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb6f01-da2d-4dc5-a5a0-1e0c0bea1548 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:45.832770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf880c-409a-4492-afa8-4471c7967734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=127c608e5ebd478282b045739182c41e, user_id=cac32da403af402c8236f46168e81038, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:45.835577 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf880c-409a-4492-afa8-4471c7967734 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:45.838184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eac787-716a-4496-b8cb-1abe1e8bd7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:45.839541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eac787-716a-4496-b8cb-1abe1e8bd7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:45.840016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eac787-716a-4496-b8cb-1abe1e8bd7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:45.840482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eac787-716a-4496-b8cb-1abe1e8bd7c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:45.841459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eac787-716a-4496-b8cb-1abe1e8bd7c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:45.843323 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eac787-716a-4496-b8cb-1abe1e8bd7c0 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:45.859649 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1017/2073] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:45 2022] PUT /identity/v3/projects/127c608e5ebd478282b045739182c41e/users/cac32da403af402c8236f46168e81038/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:45.860408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1057/2074] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:45.862932 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf880c-409a-4492-afa8-4471c7967734 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:45.866146 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12192f1d-7fc0-429a-b1a9-5c89c2944823 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:45.867523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12192f1d-7fc0-429a-b1a9-5c89c2944823 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:45.868185 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12192f1d-7fc0-429a-b1a9-5c89c2944823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:45.910460 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-12192f1d-7fc0-429a-b1a9-5c89c2944823 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:45.912768 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-12192f1d-7fc0-429a-b1a9-5c89c2944823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:46.037849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf880c-409a-4492-afa8-4471c7967734 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:46.040673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1058/2075] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:46.044899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43153001-6e23-4f21-af00-7c6d8728dc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:46.046031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43153001-6e23-4f21-af00-7c6d8728dc00 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:46.046412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43153001-6e23-4f21-af00-7c6d8728dc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:46.046784 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43153001-6e23-4f21-af00-7c6d8728dc00 admin admin] PATH_INFO: `/v3/projects/c78ff22fec8a49ed90dbe3f16c083ead/users/f899484aa5bf4336bb70c90dccea03c1/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:46.053171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12192f1d-7fc0-429a-b1a9-5c89c2944823 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:46.054694 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-12192f1d-7fc0-429a-b1a9-5c89c2944823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:46.088697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43153001-6e23-4f21-af00-7c6d8728dc00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c78ff22fec8a49ed90dbe3f16c083ead, user_id=f899484aa5bf4336bb70c90dccea03c1, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:46.091671 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43153001-6e23-4f21-af00-7c6d8728dc00 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:46.118759 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1018/2076] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:45 2022] PUT /identity/v3/projects/c78ff22fec8a49ed90dbe3f16c083ead/users/f899484aa5bf4336bb70c90dccea03c1/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:46.132609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6194e4-ef75-435a-b0c2-d36ed398b281 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:46.132960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6194e4-ef75-435a-b0c2-d36ed398b281 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:46.134126 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6194e4-ef75-435a-b0c2-d36ed398b281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:46.185955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12192f1d-7fc0-429a-b1a9-5c89c2944823 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:46.264681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-3b6194e4-ef75-435a-b0c2-d36ed398b281 None None] MFA Rules not processed for user `f899484aa5bf4336bb70c90dccea03c1`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:46.266472 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-3b6194e4-ef75-435a-b0c2-d36ed398b281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:46.358953 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a34760-f9ff-416c-a6f8-d7d0223fdb81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:46.360354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a34760-f9ff-416c-a6f8-d7d0223fdb81 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:46.363123 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a34760-f9ff-416c-a6f8-d7d0223fdb81 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:46.364085 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a34760-f9ff-416c-a6f8-d7d0223fdb81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:46.419387 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1019/2077] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:30:46.423954 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6194e4-ef75-435a-b0c2-d36ed398b281 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:46.494454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a34760-f9ff-416c-a6f8-d7d0223fdb81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:46.495993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a34760-f9ff-416c-a6f8-d7d0223fdb81 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:46.509157 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1059/2078] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:46.518321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6194e4-ef75-435a-b0c2-d36ed398b281 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:46.526366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a4da0-8564-4786-a481-4a30a414fa9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:46.527341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a4da0-8564-4786-a481-4a30a414fa9c admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:46.528345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a4da0-8564-4786-a481-4a30a414fa9c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:46.528763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a4da0-8564-4786-a481-4a30a414fa9c admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:46.529617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a4da0-8564-4786-a481-4a30a414fa9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:46.531761 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a4da0-8564-4786-a481-4a30a414fa9c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:46.589347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1020/2079] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:30:46 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:46.593175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a34760-f9ff-416c-a6f8-d7d0223fdb81 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:46.607435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a34760-f9ff-416c-a6f8-d7d0223fdb81 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:46.699702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bb438-660b-424d-9435-8529e67e9efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:46.699702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bb438-660b-424d-9435-8529e67e9efa admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:46.700012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bb438-660b-424d-9435-8529e67e9efa admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:46.700273 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bb438-660b-424d-9435-8529e67e9efa admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:46.701065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76bb438-660b-424d-9435-8529e67e9efa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:46.703726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76bb438-660b-424d-9435-8529e67e9efa admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:46.740335 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a4da0-8564-4786-a481-4a30a414fa9c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:46.740649 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1060/2080] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:30:46 2022] POST /identity/v3/users => generated 298 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:30:46.753870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a4da0-8564-4786-a481-4a30a414fa9c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:46.770003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14c575-effc-4f53-8165-a843cbfab212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:46.782546 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14c575-effc-4f53-8165-a843cbfab212 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:46.783249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14c575-effc-4f53-8165-a843cbfab212 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:46.784290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14c575-effc-4f53-8165-a843cbfab212 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:46.786378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14c575-effc-4f53-8165-a843cbfab212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:46.797744 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14c575-effc-4f53-8165-a843cbfab212 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:46.822133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1021/2081] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:30:46.830510 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bb438-660b-424d-9435-8529e67e9efa admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:46.845921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bb438-660b-424d-9435-8529e67e9efa admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:46.868099 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1398240-0aa7-4891-bdbf-20c291d56391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:46.869139 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1398240-0aa7-4891-bdbf-20c291d56391 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:46.869416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1398240-0aa7-4891-bdbf-20c291d56391 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:46.869692 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1398240-0aa7-4891-bdbf-20c291d56391 admin admin] PATH_INFO: `/v3/projects/7d1037722827455c9f3aec9e8fa1ab86/users/1a4f4d56e8cc4fe69336b8e50562a7f0/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:46.928931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1398240-0aa7-4891-bdbf-20c291d56391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1037722827455c9f3aec9e8fa1ab86, user_id=1a4f4d56e8cc4fe69336b8e50562a7f0, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:46.935921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1398240-0aa7-4891-bdbf-20c291d56391 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:46.959973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1061/2082] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:46 2022] PUT /identity/v3/projects/7d1037722827455c9f3aec9e8fa1ab86/users/1a4f4d56e8cc4fe69336b8e50562a7f0/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:30:46.964209 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14c575-effc-4f53-8165-a843cbfab212 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:46.981187 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14c575-effc-4f53-8165-a843cbfab212 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:47.139982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f514c6-3eaf-4c77-8a2b-41d30c1d5ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:47.145101 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f514c6-3eaf-4c77-8a2b-41d30c1d5ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:47.166509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f514c6-3eaf-4c77-8a2b-41d30c1d5ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:47.166886 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f514c6-3eaf-4c77-8a2b-41d30c1d5ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:47.169098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f514c6-3eaf-4c77-8a2b-41d30c1d5ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:47.172814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f514c6-3eaf-4c77-8a2b-41d30c1d5ff0 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:47.206717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1398240-0aa7-4891-bdbf-20c291d56391 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:47.207796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1022/2083] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:30:47.218756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1398240-0aa7-4891-bdbf-20c291d56391 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:47.232191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b2674-73a8-458b-992f-a56f678ce385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:47.233730 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b2674-73a8-458b-992f-a56f678ce385 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:47.234019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b2674-73a8-458b-992f-a56f678ce385 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:47.235725 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b2674-73a8-458b-992f-a56f678ce385 admin admin] PATH_INFO: `/v3/projects/7d1037722827455c9f3aec9e8fa1ab86/users/1a4f4d56e8cc4fe69336b8e50562a7f0/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:47.243959 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b2674-73a8-458b-992f-a56f678ce385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1037722827455c9f3aec9e8fa1ab86, user_id=1a4f4d56e8cc4fe69336b8e50562a7f0, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:47.247296 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b2674-73a8-458b-992f-a56f678ce385 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:47.266084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1062/2084] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:30:47 2022] PUT /identity/v3/projects/7d1037722827455c9f3aec9e8fa1ab86/users/1a4f4d56e8cc4fe69336b8e50562a7f0/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:30:47.278796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9dfdd-cef4-470f-af67-8baf8d95101b None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:47.279091 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9dfdd-cef4-470f-af67-8baf8d95101b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:47.279398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9dfdd-cef4-470f-af67-8baf8d95101b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:47.332836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-fed9dfdd-cef4-470f-af67-8baf8d95101b None None] MFA Rules not processed for user `1a4f4d56e8cc4fe69336b8e50562a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:47.334308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-fed9dfdd-cef4-470f-af67-8baf8d95101b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:47.571679 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1023/2085] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:47.587964 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b2674-73a8-458b-992f-a56f678ce385 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:47.589758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ff5b2674-73a8-458b-992f-a56f678ce385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:47.748083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b2674-73a8-458b-992f-a56f678ce385 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:47.757657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22cf66-6388-4d5a-b897-cdc71a76c43c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:47.758406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22cf66-6388-4d5a-b897-cdc71a76c43c service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:47.758687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22cf66-6388-4d5a-b897-cdc71a76c43c service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:47.758959 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22cf66-6388-4d5a-b897-cdc71a76c43c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:47.879111 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22cf66-6388-4d5a-b897-cdc71a76c43c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:47.880860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22cf66-6388-4d5a-b897-cdc71a76c43c service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:47.911084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1063/2086] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:50.089548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c14637-dfcb-4953-a7d0-762f47255375 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:50.090003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c14637-dfcb-4953-a7d0-762f47255375 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:50.090159 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c14637-dfcb-4953-a7d0-762f47255375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:50.154074 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-a1c14637-dfcb-4953-a7d0-762f47255375 None None] MFA Rules not processed for user `1be77b4735634f718f55bc6de7fd28c5`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:50.155977 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a1c14637-dfcb-4953-a7d0-762f47255375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:50.166215 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22cf66-6388-4d5a-b897-cdc71a76c43c service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:50.168813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ae22cf66-6388-4d5a-b897-cdc71a76c43c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:50.185054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22cf66-6388-4d5a-b897-cdc71a76c43c service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:50.224135 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbfa7c-8e90-4208-8d38-7d597f6a3372 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:50.224994 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbfa7c-8e90-4208-8d38-7d597f6a3372 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:50.225424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbfa7c-8e90-4208-8d38-7d597f6a3372 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:50.225846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbfa7c-8e90-4208-8d38-7d597f6a3372 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:50.315439 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1024/2087] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:50 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:50.341511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c14637-dfcb-4953-a7d0-762f47255375 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:50.343349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a1c14637-dfcb-4953-a7d0-762f47255375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:50.361811 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbfa7c-8e90-4208-8d38-7d597f6a3372 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:50.364141 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbfa7c-8e90-4208-8d38-7d597f6a3372 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:50.379347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1064/2088] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:50.472189 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c14637-dfcb-4953-a7d0-762f47255375 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:50.479518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166cb588-97aa-44ca-9395-5ab952bf0035 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:50.480438 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cb588-97aa-44ca-9395-5ab952bf0035 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:50.481892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cb588-97aa-44ca-9395-5ab952bf0035 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:50.482319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cb588-97aa-44ca-9395-5ab952bf0035 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:50.484423 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-166cb588-97aa-44ca-9395-5ab952bf0035 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:50.569514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166cb588-97aa-44ca-9395-5ab952bf0035 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:50.571126 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166cb588-97aa-44ca-9395-5ab952bf0035 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:50.586844 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1025/2089] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:51.225441 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbfa7c-8e90-4208-8d38-7d597f6a3372 service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:51.245518 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbfa7c-8e90-4208-8d38-7d597f6a3372 service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:51.264717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3215f45-94e7-4a99-a838-c561f2821a72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:51.265635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3215f45-94e7-4a99-a838-c561f2821a72 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:51.266092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3215f45-94e7-4a99-a838-c561f2821a72 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:51.266531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3215f45-94e7-4a99-a838-c561f2821a72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:51.302465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3215f45-94e7-4a99-a838-c561f2821a72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:51.304301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3215f45-94e7-4a99-a838-c561f2821a72 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:51.329436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1065/2090] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:52.301534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8e77f-f465-4119-8df6-b20c50b76f67 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:52.302420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8e77f-f465-4119-8df6-b20c50b76f67 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:52.302705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8e77f-f465-4119-8df6-b20c50b76f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:52.324451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-eda8e77f-f465-4119-8df6-b20c50b76f67 None None] MFA Rules not processed for user `96dcd82937834f89889c43a1fefc1bff`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:52.327045 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-eda8e77f-f465-4119-8df6-b20c50b76f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:52.339227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1026/2091] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:54.170524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58071e1c-c58b-4b5e-bfb4-848dfa9469b2 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:54.171500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58071e1c-c58b-4b5e-bfb4-848dfa9469b2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:54.172077 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58071e1c-c58b-4b5e-bfb4-848dfa9469b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:54.230104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-58071e1c-c58b-4b5e-bfb4-848dfa9469b2 None None] MFA Rules not processed for user `cac32da403af402c8236f46168e81038`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:54.232176 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-58071e1c-c58b-4b5e-bfb4-848dfa9469b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:54.323656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1066/2092] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:54.338604 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8e77f-f465-4119-8df6-b20c50b76f67 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:54.345415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-eda8e77f-f465-4119-8df6-b20c50b76f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:54.372428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8e77f-f465-4119-8df6-b20c50b76f67 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:54.378235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7cddb-5e59-4b4b-99cb-816ab1303759 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:54.378899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7cddb-5e59-4b4b-99cb-816ab1303759 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:54.379222 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7cddb-5e59-4b4b-99cb-816ab1303759 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:54.379500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7cddb-5e59-4b4b-99cb-816ab1303759 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:54.387644 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7cddb-5e59-4b4b-99cb-816ab1303759 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:54.389872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7cddb-5e59-4b4b-99cb-816ab1303759 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:30:54.402088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1027/2093] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:30:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:30:56.263254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126546d3-ca29-4447-940e-0f8c7219a5b3 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:56.263742 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126546d3-ca29-4447-940e-0f8c7219a5b3 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:56.263846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126546d3-ca29-4447-940e-0f8c7219a5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:56.280758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-126546d3-ca29-4447-940e-0f8c7219a5b3 None None] MFA Rules not processed for user `cac32da403af402c8236f46168e81038`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:30:56.281939 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-126546d3-ca29-4447-940e-0f8c7219a5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:30:56.288983 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1067/2094] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:30:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:30:59.823545 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7cddb-5e59-4b4b-99cb-816ab1303759 service nova] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:30:59.921984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7cddb-5e59-4b4b-99cb-816ab1303759 service nova] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:30:59.927029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c8e50-eecd-479b-97da-0d213b0f5478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:30:59.927968 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c8e50-eecd-479b-97da-0d213b0f5478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:30:59.928422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c8e50-eecd-479b-97da-0d213b0f5478 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:30:59.928878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c8e50-eecd-479b-97da-0d213b0f5478 admin admin] PATH_INFO: `/v3/users/820df11e937c4c759286828443046e52` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:30:59.930232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c8e50-eecd-479b-97da-0d213b0f5478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=820df11e937c4c759286828443046e52)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:30:59.934148 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c8e50-eecd-479b-97da-0d213b0f5478 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:00.015821 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e22c8e50-eecd-479b-97da-0d213b0f5478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '820df11e937c4c759286828443046e52'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:00.024817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e22c8e50-eecd-479b-97da-0d213b0f5478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '820df11e937c4c759286828443046e52'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:00.025779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e22c8e50-eecd-479b-97da-0d213b0f5478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '820df11e937c4c759286828443046e52'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:00.036408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e22c8e50-eecd-479b-97da-0d213b0f5478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '820df11e937c4c759286828443046e52'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:00.037034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e22c8e50-eecd-479b-97da-0d213b0f5478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '820df11e937c4c759286828443046e52'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:00.068594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1028/2095] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:30:59 2022] DELETE /identity/v3/users/820df11e937c4c759286828443046e52 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:00.070508 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c3395-8beb-4285-b1fb-4be017f516bb None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:00.070952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c3395-8beb-4285-b1fb-4be017f516bb None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:00.070952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c3395-8beb-4285-b1fb-4be017f516bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:00.098808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-354c3395-8beb-4285-b1fb-4be017f516bb None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:00.100445 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-354c3395-8beb-4285-b1fb-4be017f516bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:00.213476 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1068/2096] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:31:00.220196 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c8e50-eecd-479b-97da-0d213b0f5478 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:00.222330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e22c8e50-eecd-479b-97da-0d213b0f5478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:00.309593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c8e50-eecd-479b-97da-0d213b0f5478 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:00.314552 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ffcf8-29c2-4583-ad7e-aa1fb5dbc251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:00.315534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ffcf8-29c2-4583-ad7e-aa1fb5dbc251 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:00.316216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ffcf8-29c2-4583-ad7e-aa1fb5dbc251 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:00.316467 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ffcf8-29c2-4583-ad7e-aa1fb5dbc251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:00.333188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ffcf8-29c2-4583-ad7e-aa1fb5dbc251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:00.335112 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ffcf8-29c2-4583-ad7e-aa1fb5dbc251 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:00.352937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1029/2097] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:31:00.862582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c3395-8beb-4285-b1fb-4be017f516bb None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:00.864308 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-354c3395-8beb-4285-b1fb-4be017f516bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:00.886411 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c3395-8beb-4285-b1fb-4be017f516bb None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:00.899391 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:00.900505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:00.900948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:00.902048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] PATH_INFO: `/v3/projects/c31fb870496946e18fa3e115da89ae62` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:00.904003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c31fb870496946e18fa3e115da89ae62)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:00.906078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:00.969749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31fb870496946e18fa3e115da89ae62', 'request_id': 'req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:00.978778 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31fb870496946e18fa3e115da89ae62', 'request_id': 'req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:00.989330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31fb870496946e18fa3e115da89ae62', 'request_id': 'req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.003216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] The token cache is being invalidate because project c31fb870496946e18fa3e115da89ae62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:31:01.003607 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.004307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.006267 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1069/2098] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:31:00 2022] DELETE /identity/v3/projects/c31fb870496946e18fa3e115da89ae62 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:31:01.011398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ffcf8-29c2-4583-ad7e-aa1fb5dbc251 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:01.024458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ffcf8-29c2-4583-ad7e-aa1fb5dbc251 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:01.089007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92eef07-fa8a-4e4b-87e1-326af0f76040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:01.090643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eef07-fa8a-4e4b-87e1-326af0f76040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:01.090981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eef07-fa8a-4e4b-87e1-326af0f76040 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:01.091303 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eef07-fa8a-4e4b-87e1-326af0f76040 admin admin] PATH_INFO: `/v3/users/f899484aa5bf4336bb70c90dccea03c1` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:01.092744 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92eef07-fa8a-4e4b-87e1-326af0f76040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f899484aa5bf4336bb70c90dccea03c1)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:01.095213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92eef07-fa8a-4e4b-87e1-326af0f76040 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:01.148067 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b92eef07-fa8a-4e4b-87e1-326af0f76040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f899484aa5bf4336bb70c90dccea03c1'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.157421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b92eef07-fa8a-4e4b-87e1-326af0f76040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f899484aa5bf4336bb70c90dccea03c1'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.158211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b92eef07-fa8a-4e4b-87e1-326af0f76040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f899484aa5bf4336bb70c90dccea03c1'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.166667 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b92eef07-fa8a-4e4b-87e1-326af0f76040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f899484aa5bf4336bb70c90dccea03c1'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.167413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b92eef07-fa8a-4e4b-87e1-326af0f76040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f899484aa5bf4336bb70c90dccea03c1'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.191535 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1030/2099] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:31:01 2022] DELETE /identity/v3/users/f899484aa5bf4336bb70c90dccea03c1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:01.466929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:01.468503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:01.483203 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03c3eb-4aab-4caf-9dbd-7db6adfe6361 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:01.559680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:01.560552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:01.560825 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:01.561092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] PATH_INFO: `/v3/projects/c78ff22fec8a49ed90dbe3f16c083ead` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:01.562255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c78ff22fec8a49ed90dbe3f16c083ead)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:01.564200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:01.615665 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c78ff22fec8a49ed90dbe3f16c083ead', 'request_id': 'req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.623033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c78ff22fec8a49ed90dbe3f16c083ead', 'request_id': 'req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.633471 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c78ff22fec8a49ed90dbe3f16c083ead', 'request_id': 'req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.658090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] The token cache is being invalidate because project c78ff22fec8a49ed90dbe3f16c083ead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:31:01.658475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.659269 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.661389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1070/2100] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:31:01 2022] DELETE /identity/v3/projects/c78ff22fec8a49ed90dbe3f16c083ead => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:01.665786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92eef07-fa8a-4e4b-87e1-326af0f76040 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:01.672314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b92eef07-fa8a-4e4b-87e1-326af0f76040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:01.685610 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92eef07-fa8a-4e4b-87e1-326af0f76040 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:01.749020 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969d8ca-cbed-4bbb-8a6c-171ae36f5de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:01.749957 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969d8ca-cbed-4bbb-8a6c-171ae36f5de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:01.750238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969d8ca-cbed-4bbb-8a6c-171ae36f5de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:01.750509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969d8ca-cbed-4bbb-8a6c-171ae36f5de7 admin admin] PATH_INFO: `/v3/users/1a4f4d56e8cc4fe69336b8e50562a7f0` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:01.752090 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969d8ca-cbed-4bbb-8a6c-171ae36f5de7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a4f4d56e8cc4fe69336b8e50562a7f0)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:01.754080 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969d8ca-cbed-4bbb-8a6c-171ae36f5de7 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:01.802473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2969d8ca-cbed-4bbb-8a6c-171ae36f5de7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a4f4d56e8cc4fe69336b8e50562a7f0'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.814784 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2969d8ca-cbed-4bbb-8a6c-171ae36f5de7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a4f4d56e8cc4fe69336b8e50562a7f0'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.816146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2969d8ca-cbed-4bbb-8a6c-171ae36f5de7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a4f4d56e8cc4fe69336b8e50562a7f0'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.827145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2969d8ca-cbed-4bbb-8a6c-171ae36f5de7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a4f4d56e8cc4fe69336b8e50562a7f0'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.827870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-2969d8ca-cbed-4bbb-8a6c-171ae36f5de7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a4f4d56e8cc4fe69336b8e50562a7f0'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:01.849385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1031/2101] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:31:01 2022] DELETE /identity/v3/users/1a4f4d56e8cc4fe69336b8e50562a7f0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:31:02.234317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:02.236273 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:02.258471 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36dd2b8-b72e-4c3b-bf54-b8d725c1b0b5 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:02.315859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d97a4-cd69-426d-adee-0587b5451b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:02.316651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d97a4-cd69-426d-adee-0587b5451b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:02.316952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d97a4-cd69-426d-adee-0587b5451b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:02.317233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d97a4-cd69-426d-adee-0587b5451b76 admin admin] PATH_INFO: `/v3/projects/7d1037722827455c9f3aec9e8fa1ab86` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:02.318436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d97a4-cd69-426d-adee-0587b5451b76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d1037722827455c9f3aec9e8fa1ab86)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:02.320531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d97a4-cd69-426d-adee-0587b5451b76 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:02.366227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-081d97a4-cd69-426d-adee-0587b5451b76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d1037722827455c9f3aec9e8fa1ab86', 'request_id': 'req-081d97a4-cd69-426d-adee-0587b5451b76'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:02.371762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-081d97a4-cd69-426d-adee-0587b5451b76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d1037722827455c9f3aec9e8fa1ab86', 'request_id': 'req-081d97a4-cd69-426d-adee-0587b5451b76'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:02.379153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-081d97a4-cd69-426d-adee-0587b5451b76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d1037722827455c9f3aec9e8fa1ab86', 'request_id': 'req-081d97a4-cd69-426d-adee-0587b5451b76'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:02.389806 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-081d97a4-cd69-426d-adee-0587b5451b76 admin admin] The token cache is being invalidate because project 7d1037722827455c9f3aec9e8fa1ab86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:31:02.390089 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-081d97a4-cd69-426d-adee-0587b5451b76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:02.390710 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-081d97a4-cd69-426d-adee-0587b5451b76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:02.392260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1071/2102] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:31:02 2022] DELETE /identity/v3/projects/7d1037722827455c9f3aec9e8fa1ab86 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:02.397968 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797bd2b-255f-419a-b4c9-cd198f15244d None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:02.398439 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797bd2b-255f-419a-b4c9-cd198f15244d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:02.398534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797bd2b-255f-419a-b4c9-cd198f15244d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:02.418529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-1797bd2b-255f-419a-b4c9-cd198f15244d None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:02.419768 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1797bd2b-255f-419a-b4c9-cd198f15244d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:02.476240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1032/2103] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:02.484852 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe21e0-3c37-4e3d-affb-bb5d6ccca474 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:02.485332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe21e0-3c37-4e3d-affb-bb5d6ccca474 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:02.485426 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe21e0-3c37-4e3d-affb-bb5d6ccca474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:02.503308 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4bfe21e0-3c37-4e3d-affb-bb5d6ccca474 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:02.504489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4bfe21e0-3c37-4e3d-affb-bb5d6ccca474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:02.512650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1072/2104] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:02.516951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797bd2b-255f-419a-b4c9-cd198f15244d None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:02.527916 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797bd2b-255f-419a-b4c9-cd198f15244d None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:02.532513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaa3e2-5c61-44d4-adc6-71fd22aa490a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:02.533202 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaa3e2-5c61-44d4-adc6-71fd22aa490a admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:02.533478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaa3e2-5c61-44d4-adc6-71fd22aa490a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:02.533750 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaa3e2-5c61-44d4-adc6-71fd22aa490a admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:02.534471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaa3e2-5c61-44d4-adc6-71fd22aa490a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:31:02.534781 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaa3e2-5c61-44d4-adc6-71fd22aa490a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:02.536408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaa3e2-5c61-44d4-adc6-71fd22aa490a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:02.548597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1033/2105] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:31:02.552888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe21e0-3c37-4e3d-affb-bb5d6ccca474 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:02.562002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe21e0-3c37-4e3d-affb-bb5d6ccca474 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:02.566958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c683588-fe54-49dc-86f1-578a9ea0a3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:02.567585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c683588-fe54-49dc-86f1-578a9ea0a3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:02.567894 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c683588-fe54-49dc-86f1-578a9ea0a3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:02.568292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c683588-fe54-49dc-86f1-578a9ea0a3a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:02.568897 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c683588-fe54-49dc-86f1-578a9ea0a3a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:02.570853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c683588-fe54-49dc-86f1-578a9ea0a3a4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:02.597997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1073/2106] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:31:02 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:02.602038 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaa3e2-5c61-44d4-adc6-71fd22aa490a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:02.611917 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaa3e2-5c61-44d4-adc6-71fd22aa490a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:02.670624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08073306-b103-477e-be7c-3299a048adf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:02.671333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08073306-b103-477e-be7c-3299a048adf0 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:02.671615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08073306-b103-477e-be7c-3299a048adf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:02.671887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08073306-b103-477e-be7c-3299a048adf0 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:02.672575 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08073306-b103-477e-be7c-3299a048adf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:02.674482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08073306-b103-477e-be7c-3299a048adf0 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:02.697822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1034/2107] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:31:02 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:02.698798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c683588-fe54-49dc-86f1-578a9ea0a3a4 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:02.709363 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c683588-fe54-49dc-86f1-578a9ea0a3a4 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:02.715511 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c257b3-0d53-44ff-8d93-68ce8fe700ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:02.716226 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c257b3-0d53-44ff-8d93-68ce8fe700ec admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:02.716509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c257b3-0d53-44ff-8d93-68ce8fe700ec admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:02.716940 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c257b3-0d53-44ff-8d93-68ce8fe700ec admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:02.717673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c257b3-0d53-44ff-8d93-68ce8fe700ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:02.719742 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c257b3-0d53-44ff-8d93-68ce8fe700ec admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:02.731337 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1074/2108] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:31:02.734617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08073306-b103-477e-be7c-3299a048adf0 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:02.744693 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08073306-b103-477e-be7c-3299a048adf0 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:02.749396 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb693b-c3d2-4090-93c8-ece79ea3dc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:02.750062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb693b-c3d2-4090-93c8-ece79ea3dc9b admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:02.750351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb693b-c3d2-4090-93c8-ece79ea3dc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:02.750629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb693b-c3d2-4090-93c8-ece79ea3dc9b admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:02.751389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb693b-c3d2-4090-93c8-ece79ea3dc9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:02.753246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb693b-c3d2-4090-93c8-ece79ea3dc9b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:02.764051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1035/2109] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:31:02.768527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c257b3-0d53-44ff-8d93-68ce8fe700ec admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:02.779434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c257b3-0d53-44ff-8d93-68ce8fe700ec admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:02.784154 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e68c65-55e5-4c87-993f-324666f4d568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:02.785040 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e68c65-55e5-4c87-993f-324666f4d568 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:02.785326 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e68c65-55e5-4c87-993f-324666f4d568 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:02.785600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e68c65-55e5-4c87-993f-324666f4d568 admin admin] PATH_INFO: `/v3/projects/56fcc79de10140149d58f55ac418a103/users/529c3f9f00704b198708d77d0a49b8fc/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:02.799554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e68c65-55e5-4c87-993f-324666f4d568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56fcc79de10140149d58f55ac418a103, user_id=529c3f9f00704b198708d77d0a49b8fc, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:02.802256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e68c65-55e5-4c87-993f-324666f4d568 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:02.815137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1075/2110] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:31:02 2022] PUT /identity/v3/projects/56fcc79de10140149d58f55ac418a103/users/529c3f9f00704b198708d77d0a49b8fc/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:02.826181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba756369-4fae-4354-aaa5-386d9150aae9 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:02.826407 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba756369-4fae-4354-aaa5-386d9150aae9 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:02.826833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba756369-4fae-4354-aaa5-386d9150aae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:02.855953 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ba756369-4fae-4354-aaa5-386d9150aae9 None None] MFA Rules not processed for user `529c3f9f00704b198708d77d0a49b8fc`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:02.857218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ba756369-4fae-4354-aaa5-386d9150aae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:02.925502 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1036/2111] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:02.937618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e68c65-55e5-4c87-993f-324666f4d568 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:02.939411 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-d8e68c65-55e5-4c87-993f-324666f4d568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:03.002795 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e68c65-55e5-4c87-993f-324666f4d568 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:03.006451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07134dd8-d47f-4e7f-8720-c9eeb71d0a03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:03.007102 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07134dd8-d47f-4e7f-8720-c9eeb71d0a03 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:03.007414 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07134dd8-d47f-4e7f-8720-c9eeb71d0a03 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:03.007690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07134dd8-d47f-4e7f-8720-c9eeb71d0a03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:03.017220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07134dd8-d47f-4e7f-8720-c9eeb71d0a03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:03.018643 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07134dd8-d47f-4e7f-8720-c9eeb71d0a03 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:03.027691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1076/2112] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:31:05.118430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba756369-4fae-4354-aaa5-386d9150aae9 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:05.119854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ba756369-4fae-4354-aaa5-386d9150aae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:05.187248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba756369-4fae-4354-aaa5-386d9150aae9 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:05.191698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f9539-dd8a-431a-b50b-80bf3f85c94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:05.192573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f9539-dd8a-431a-b50b-80bf3f85c94a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:05.192863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f9539-dd8a-431a-b50b-80bf3f85c94a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:05.193139 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f9539-dd8a-431a-b50b-80bf3f85c94a admin admin] PATH_INFO: `/v3/users/b42135b59eb443b1964b1eb579315153` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:05.194311 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f9539-dd8a-431a-b50b-80bf3f85c94a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b42135b59eb443b1964b1eb579315153)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:05.196258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f9539-dd8a-431a-b50b-80bf3f85c94a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:05.242176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-431f9539-dd8a-431a-b50b-80bf3f85c94a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b42135b59eb443b1964b1eb579315153'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.249459 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-431f9539-dd8a-431a-b50b-80bf3f85c94a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b42135b59eb443b1964b1eb579315153'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.250116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-431f9539-dd8a-431a-b50b-80bf3f85c94a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b42135b59eb443b1964b1eb579315153'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.257390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-431f9539-dd8a-431a-b50b-80bf3f85c94a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b42135b59eb443b1964b1eb579315153'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.258149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-431f9539-dd8a-431a-b50b-80bf3f85c94a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b42135b59eb443b1964b1eb579315153'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.276590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1037/2113] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:31:05 2022] DELETE /identity/v3/users/b42135b59eb443b1964b1eb579315153 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:05.281137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3179eb8-0c5d-4950-a4e4-427a75489423 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:05.281609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3179eb8-0c5d-4950-a4e4-427a75489423 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:05.281698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3179eb8-0c5d-4950-a4e4-427a75489423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:05.297750 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b3179eb8-0c5d-4950-a4e4-427a75489423 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:05.298910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b3179eb8-0c5d-4950-a4e4-427a75489423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:05.357582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1077/2114] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:05.367049 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f9539-dd8a-431a-b50b-80bf3f85c94a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:05.368439 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-431f9539-dd8a-431a-b50b-80bf3f85c94a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:05.425564 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f9539-dd8a-431a-b50b-80bf3f85c94a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:05.429142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f462e-8366-46a0-88be-a2905ed2735b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:05.429789 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f462e-8366-46a0-88be-a2905ed2735b service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:05.430071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f462e-8366-46a0-88be-a2905ed2735b service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:05.430346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f462e-8366-46a0-88be-a2905ed2735b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:05.439678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f462e-8366-46a0-88be-a2905ed2735b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:05.441087 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f462e-8366-46a0-88be-a2905ed2735b service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:05.450813 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1038/2115] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:31:05.603208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3179eb8-0c5d-4950-a4e4-427a75489423 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:05.604633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b3179eb8-0c5d-4950-a4e4-427a75489423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:05.621756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3179eb8-0c5d-4950-a4e4-427a75489423 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:05.632914 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c401c8e-6a01-4553-9c8e-d0fea223c3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:05.634049 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c401c8e-6a01-4553-9c8e-d0fea223c3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:05.635108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c401c8e-6a01-4553-9c8e-d0fea223c3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:05.635382 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c401c8e-6a01-4553-9c8e-d0fea223c3e3 admin admin] PATH_INFO: `/v3/users/529c3f9f00704b198708d77d0a49b8fc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:05.637101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c401c8e-6a01-4553-9c8e-d0fea223c3e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=529c3f9f00704b198708d77d0a49b8fc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:05.639824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c401c8e-6a01-4553-9c8e-d0fea223c3e3 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:05.696495 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9c401c8e-6a01-4553-9c8e-d0fea223c3e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '529c3f9f00704b198708d77d0a49b8fc'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.697138 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9c401c8e-6a01-4553-9c8e-d0fea223c3e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '529c3f9f00704b198708d77d0a49b8fc'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.704718 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9c401c8e-6a01-4553-9c8e-d0fea223c3e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '529c3f9f00704b198708d77d0a49b8fc'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.714140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9c401c8e-6a01-4553-9c8e-d0fea223c3e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '529c3f9f00704b198708d77d0a49b8fc'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.714933 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9c401c8e-6a01-4553-9c8e-d0fea223c3e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '529c3f9f00704b198708d77d0a49b8fc'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.732394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1078/2116] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:31:05 2022] DELETE /identity/v3/users/529c3f9f00704b198708d77d0a49b8fc => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:05.737134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dae8d9-74d3-4d39-9046-e5363d43daf1 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:05.737586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dae8d9-74d3-4d39-9046-e5363d43daf1 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:05.738039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dae8d9-74d3-4d39-9046-e5363d43daf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:05.761545 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c401c8e-6a01-4553-9c8e-d0fea223c3e3 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:05.762114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-77dae8d9-74d3-4d39-9046-e5363d43daf1 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:05.763373 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-77dae8d9-74d3-4d39-9046-e5363d43daf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:05.763849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9c401c8e-6a01-4553-9c8e-d0fea223c3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:05.833257 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1039/2117] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:05.842436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c401c8e-6a01-4553-9c8e-d0fea223c3e3 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:05.842894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dae8d9-74d3-4d39-9046-e5363d43daf1 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:05.844486 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-77dae8d9-74d3-4d39-9046-e5363d43daf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:05.847628 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95309f8-0ee6-4073-80c5-2965b99cbd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:05.848756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95309f8-0ee6-4073-80c5-2965b99cbd54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:05.853598 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95309f8-0ee6-4073-80c5-2965b99cbd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:05.853909 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95309f8-0ee6-4073-80c5-2965b99cbd54 admin admin] PATH_INFO: `/v3/projects/5ccc781135024a86a719aab2736034ef` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:05.855924 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95309f8-0ee6-4073-80c5-2965b99cbd54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ccc781135024a86a719aab2736034ef)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:05.857707 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95309f8-0ee6-4073-80c5-2965b99cbd54 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:05.906318 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f95309f8-0ee6-4073-80c5-2965b99cbd54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ccc781135024a86a719aab2736034ef', 'request_id': 'req-f95309f8-0ee6-4073-80c5-2965b99cbd54'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.913084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f95309f8-0ee6-4073-80c5-2965b99cbd54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ccc781135024a86a719aab2736034ef', 'request_id': 'req-f95309f8-0ee6-4073-80c5-2965b99cbd54'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.914369 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dae8d9-74d3-4d39-9046-e5363d43daf1 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:05.925261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f95309f8-0ee6-4073-80c5-2965b99cbd54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ccc781135024a86a719aab2736034ef', 'request_id': 'req-f95309f8-0ee6-4073-80c5-2965b99cbd54'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.941489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f95309f8-0ee6-4073-80c5-2965b99cbd54 admin admin] The token cache is being invalidate because project 5ccc781135024a86a719aab2736034ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:31:05.941854 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f95309f8-0ee6-4073-80c5-2965b99cbd54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.942594 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f95309f8-0ee6-4073-80c5-2965b99cbd54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:05.944538 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1079/2118] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:31:05 2022] DELETE /identity/v3/projects/5ccc781135024a86a719aab2736034ef => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:05.949166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f0bcd-eb32-4ef5-b43d-07ca2407be67 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:05.949527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f0bcd-eb32-4ef5-b43d-07ca2407be67 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:05.949876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f0bcd-eb32-4ef5-b43d-07ca2407be67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:05.976439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-bf8f0bcd-eb32-4ef5-b43d-07ca2407be67 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:05.977248 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-bf8f0bcd-eb32-4ef5-b43d-07ca2407be67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:05.977606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873b90a-5cbf-4063-a53a-42a35a0b54f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:05.978277 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873b90a-5cbf-4063-a53a-42a35a0b54f5 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:05.978558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873b90a-5cbf-4063-a53a-42a35a0b54f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:05.978835 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873b90a-5cbf-4063-a53a-42a35a0b54f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:05.984865 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c873b90a-5cbf-4063-a53a-42a35a0b54f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:06.049831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1080/2119] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:31:06.054050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873b90a-5cbf-4063-a53a-42a35a0b54f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.055497 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873b90a-5cbf-4063-a53a-42a35a0b54f5 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.058098 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fd77d-1498-4f74-a6e0-7de16fefd768 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.058427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fd77d-1498-4f74-a6e0-7de16fefd768 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.058702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fd77d-1498-4f74-a6e0-7de16fefd768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.065424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1040/2120] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:31:06.077764 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b22fd77d-1498-4f74-a6e0-7de16fefd768 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:06.079129 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b22fd77d-1498-4f74-a6e0-7de16fefd768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:06.087081 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1081/2121] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:31:06.091817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873b90a-5cbf-4063-a53a-42a35a0b54f5 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:06.101780 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873b90a-5cbf-4063-a53a-42a35a0b54f5 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:06.107106 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e7ace-bcdc-45dc-97bd-6af999fb1d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:06.107824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e7ace-bcdc-45dc-97bd-6af999fb1d6b admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.108103 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e7ace-bcdc-45dc-97bd-6af999fb1d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.108375 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e7ace-bcdc-45dc-97bd-6af999fb1d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.109103 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e7ace-bcdc-45dc-97bd-6af999fb1d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:31:06.109403 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e7ace-bcdc-45dc-97bd-6af999fb1d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.111145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e7ace-bcdc-45dc-97bd-6af999fb1d6b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.125391 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1041/2122] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:31:06.130300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fd77d-1498-4f74-a6e0-7de16fefd768 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:06.139689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fd77d-1498-4f74-a6e0-7de16fefd768 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:06.144533 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c23f7-0e29-4bde-a81b-739a25a67350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:06.145192 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c23f7-0e29-4bde-a81b-739a25a67350 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.145472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c23f7-0e29-4bde-a81b-739a25a67350 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.145747 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c23f7-0e29-4bde-a81b-739a25a67350 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.146437 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c23f7-0e29-4bde-a81b-739a25a67350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.148358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c23f7-0e29-4bde-a81b-739a25a67350 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.182552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1082/2123] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:31:06 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:31:06.183859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e7ace-bcdc-45dc-97bd-6af999fb1d6b admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:06.193679 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e7ace-bcdc-45dc-97bd-6af999fb1d6b admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:06.260221 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9004c-b438-483a-909b-875acdc89b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:06.261006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9004c-b438-483a-909b-875acdc89b2a admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.261742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9004c-b438-483a-909b-875acdc89b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.262220 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9004c-b438-483a-909b-875acdc89b2a admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.262981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9004c-b438-483a-909b-875acdc89b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.265079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9004c-b438-483a-909b-875acdc89b2a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.288622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1042/2124] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:31:06 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:06.289787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c23f7-0e29-4bde-a81b-739a25a67350 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:06.301472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c23f7-0e29-4bde-a81b-739a25a67350 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:06.306325 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae129e-bdf4-4500-ba7d-bbaa4e7f8e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:06.307006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae129e-bdf4-4500-ba7d-bbaa4e7f8e08 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.307327 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae129e-bdf4-4500-ba7d-bbaa4e7f8e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.307606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae129e-bdf4-4500-ba7d-bbaa4e7f8e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.308401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae129e-bdf4-4500-ba7d-bbaa4e7f8e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.310017 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae129e-bdf4-4500-ba7d-bbaa4e7f8e08 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.320582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1083/2125] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:31:06.325223 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9004c-b438-483a-909b-875acdc89b2a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:06.335704 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9004c-b438-483a-909b-875acdc89b2a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:06.341677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512760c0-dd5f-40cf-9887-8801ea3bf1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:06.342525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512760c0-dd5f-40cf-9887-8801ea3bf1b2 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.342936 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512760c0-dd5f-40cf-9887-8801ea3bf1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.343252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512760c0-dd5f-40cf-9887-8801ea3bf1b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.344050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512760c0-dd5f-40cf-9887-8801ea3bf1b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.346051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512760c0-dd5f-40cf-9887-8801ea3bf1b2 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.360403 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1043/2126] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:31:06.362090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae129e-bdf4-4500-ba7d-bbaa4e7f8e08 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:06.362229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512760c0-dd5f-40cf-9887-8801ea3bf1b2 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:06.364031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-feae129e-bdf4-4500-ba7d-bbaa4e7f8e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:06.375435 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512760c0-dd5f-40cf-9887-8801ea3bf1b2 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:06.377514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae129e-bdf4-4500-ba7d-bbaa4e7f8e08 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:06.380695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7d82b-7645-40b5-ad41-fee874e3ac24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:06.381976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7d82b-7645-40b5-ad41-fee874e3ac24 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.381976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7d82b-7645-40b5-ad41-fee874e3ac24 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.382183 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7d82b-7645-40b5-ad41-fee874e3ac24 admin admin] PATH_INFO: `/v3/projects/a2b375defd1e41149b8949a808b49dd0/users/abae69a93a014143ab0cebfd1a7be830/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.382969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d121a-8110-457b-9f62-05f76bfdb21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:06.384039 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d121a-8110-457b-9f62-05f76bfdb21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.384441 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d121a-8110-457b-9f62-05f76bfdb21b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.384860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d121a-8110-457b-9f62-05f76bfdb21b admin admin] PATH_INFO: `/v3/projects/56fcc79de10140149d58f55ac418a103` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.387096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d121a-8110-457b-9f62-05f76bfdb21b admin admin] RBAC: Authorizing `identity:delete_project(project_id=56fcc79de10140149d58f55ac418a103)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.389030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d121a-8110-457b-9f62-05f76bfdb21b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.400130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7d82b-7645-40b5-ad41-fee874e3ac24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b375defd1e41149b8949a808b49dd0, user_id=abae69a93a014143ab0cebfd1a7be830, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.403481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7d82b-7645-40b5-ad41-fee874e3ac24 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.417185 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1044/2127] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:31:06 2022] PUT /identity/v3/projects/a2b375defd1e41149b8949a808b49dd0/users/abae69a93a014143ab0cebfd1a7be830/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:06.427524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f2bcd-bd1f-42d0-bb47-8a1ec91ea3f4 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.427819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f2bcd-bd1f-42d0-bb47-8a1ec91ea3f4 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.428098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f2bcd-bd1f-42d0-bb47-8a1ec91ea3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.448178 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-883d121a-8110-457b-9f62-05f76bfdb21b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56fcc79de10140149d58f55ac418a103', 'request_id': 'req-883d121a-8110-457b-9f62-05f76bfdb21b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:06.454130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-883d121a-8110-457b-9f62-05f76bfdb21b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56fcc79de10140149d58f55ac418a103', 'request_id': 'req-883d121a-8110-457b-9f62-05f76bfdb21b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:06.455858 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f93f2bcd-bd1f-42d0-bb47-8a1ec91ea3f4 None None] MFA Rules not processed for user `abae69a93a014143ab0cebfd1a7be830`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:06.457088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f93f2bcd-bd1f-42d0-bb47-8a1ec91ea3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:06.461653 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-883d121a-8110-457b-9f62-05f76bfdb21b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56fcc79de10140149d58f55ac418a103', 'request_id': 'req-883d121a-8110-457b-9f62-05f76bfdb21b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:06.473497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-883d121a-8110-457b-9f62-05f76bfdb21b admin admin] The token cache is being invalidate because project 56fcc79de10140149d58f55ac418a103 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:31:06.473868 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-883d121a-8110-457b-9f62-05f76bfdb21b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:06.474617 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-883d121a-8110-457b-9f62-05f76bfdb21b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:06.476269 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1084/2128] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:31:06 2022] DELETE /identity/v3/projects/56fcc79de10140149d58f55ac418a103 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:06.480918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201b444-a8e2-4540-a943-4fb0559c95f0 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.481206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201b444-a8e2-4540-a943-4fb0559c95f0 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.481486 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201b444-a8e2-4540-a943-4fb0559c95f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.499451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4201b444-a8e2-4540-a943-4fb0559c95f0 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:06.500648 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4201b444-a8e2-4540-a943-4fb0559c95f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:06.526639 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1045/2129] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:31:06.540260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f2bcd-bd1f-42d0-bb47-8a1ec91ea3f4 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:06.541662 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f93f2bcd-bd1f-42d0-bb47-8a1ec91ea3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:06.560117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1085/2130] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:31:06.568613 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48524d46-e24a-4764-807f-a7a272665295 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.568902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48524d46-e24a-4764-807f-a7a272665295 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.569184 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48524d46-e24a-4764-807f-a7a272665295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.587663 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-48524d46-e24a-4764-807f-a7a272665295 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:06.589005 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-48524d46-e24a-4764-807f-a7a272665295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:06.597944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1086/2131] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:31:06.602253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48524d46-e24a-4764-807f-a7a272665295 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:06.603429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f2bcd-bd1f-42d0-bb47-8a1ec91ea3f4 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:06.607190 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af49929-73a2-4bdd-b110-4e83e7cbee3d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cf284e81ba4042b66c0bc189f1ae2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:06.607866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af49929-73a2-4bdd-b110-4e83e7cbee3d service glance] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.608144 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af49929-73a2-4bdd-b110-4e83e7cbee3d service glance] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.608572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af49929-73a2-4bdd-b110-4e83e7cbee3d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.610994 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48524d46-e24a-4764-807f-a7a272665295 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:06.615865 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d37364-bb2b-4f34-a8e1-79a435c11920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:06.616511 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d37364-bb2b-4f34-a8e1-79a435c11920 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.616788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d37364-bb2b-4f34-a8e1-79a435c11920 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.616956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af49929-73a2-4bdd-b110-4e83e7cbee3d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.617068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d37364-bb2b-4f34-a8e1-79a435c11920 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.617771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d37364-bb2b-4f34-a8e1-79a435c11920 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:31:06.618081 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d37364-bb2b-4f34-a8e1-79a435c11920 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.618384 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af49929-73a2-4bdd-b110-4e83e7cbee3d service glance] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.619738 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d37364-bb2b-4f34-a8e1-79a435c11920 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.628489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1046/2132] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:31:06.634423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1087/2133] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:31:06.639884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af49929-73a2-4bdd-b110-4e83e7cbee3d service glance] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:06.651336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af49929-73a2-4bdd-b110-4e83e7cbee3d service glance] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:06.657280 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86014bcf-ebd9-4712-a653-c5fd99c9c162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:06.658074 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86014bcf-ebd9-4712-a653-c5fd99c9c162 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.658354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86014bcf-ebd9-4712-a653-c5fd99c9c162 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.658624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86014bcf-ebd9-4712-a653-c5fd99c9c162 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.659345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86014bcf-ebd9-4712-a653-c5fd99c9c162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.661281 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86014bcf-ebd9-4712-a653-c5fd99c9c162 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.691740 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1047/2134] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:31:06 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:06.694760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d37364-bb2b-4f34-a8e1-79a435c11920 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:06.706101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d37364-bb2b-4f34-a8e1-79a435c11920 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:06.771498 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b3e9a-f5aa-4b65-b77d-9d66f1469d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:06.772966 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b3e9a-f5aa-4b65-b77d-9d66f1469d5b admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.772966 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b3e9a-f5aa-4b65-b77d-9d66f1469d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.773185 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b3e9a-f5aa-4b65-b77d-9d66f1469d5b admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.773933 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b3e9a-f5aa-4b65-b77d-9d66f1469d5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.776030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b3e9a-f5aa-4b65-b77d-9d66f1469d5b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.800701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1088/2135] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:31:06 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:06.804735 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86014bcf-ebd9-4712-a653-c5fd99c9c162 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:06.816013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86014bcf-ebd9-4712-a653-c5fd99c9c162 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:06.821099 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440979d-52f4-4aca-a7b6-7f4a1c34ca48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:06.821882 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440979d-52f4-4aca-a7b6-7f4a1c34ca48 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.822269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440979d-52f4-4aca-a7b6-7f4a1c34ca48 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.822614 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440979d-52f4-4aca-a7b6-7f4a1c34ca48 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.823503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440979d-52f4-4aca-a7b6-7f4a1c34ca48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.826206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440979d-52f4-4aca-a7b6-7f4a1c34ca48 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.839467 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1048/2136] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:31:06.843376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b3e9a-f5aa-4b65-b77d-9d66f1469d5b admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:06.853379 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b3e9a-f5aa-4b65-b77d-9d66f1469d5b admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:06.863845 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c782ee6-7233-41d0-b479-9abaec31ea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:06.864777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782ee6-7233-41d0-b479-9abaec31ea8f admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.865171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782ee6-7233-41d0-b479-9abaec31ea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.865455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782ee6-7233-41d0-b479-9abaec31ea8f admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.866292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c782ee6-7233-41d0-b479-9abaec31ea8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.868031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c782ee6-7233-41d0-b479-9abaec31ea8f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.891619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1089/2137] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:31:06.896842 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440979d-52f4-4aca-a7b6-7f4a1c34ca48 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:06.907413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440979d-52f4-4aca-a7b6-7f4a1c34ca48 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:06.912528 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28375f0-f12d-42c9-a82c-e1ff5b4d106e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:06.913412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28375f0-f12d-42c9-a82c-e1ff5b4d106e admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.913694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28375f0-f12d-42c9-a82c-e1ff5b4d106e admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.913963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28375f0-f12d-42c9-a82c-e1ff5b4d106e admin admin] PATH_INFO: `/v3/projects/bc8ecae09da549248c002d344f413ed2/users/96cdbb1be17c4352ba93c31238c8b96b/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:06.930574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28375f0-f12d-42c9-a82c-e1ff5b4d106e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8ecae09da549248c002d344f413ed2, user_id=96cdbb1be17c4352ba93c31238c8b96b, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:06.936938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28375f0-f12d-42c9-a82c-e1ff5b4d106e admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:06.963855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1049/2138] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:31:06 2022] PUT /identity/v3/projects/bc8ecae09da549248c002d344f413ed2/users/96cdbb1be17c4352ba93c31238c8b96b/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:06.965581 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e9559-cd30-4d1a-be82-ae75f7b1cde9 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:06.965876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e9559-cd30-4d1a-be82-ae75f7b1cde9 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:06.966179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e9559-cd30-4d1a-be82-ae75f7b1cde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:07.004921 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-345e9559-cd30-4d1a-be82-ae75f7b1cde9 None None] MFA Rules not processed for user `96cdbb1be17c4352ba93c31238c8b96b`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:07.006202 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-345e9559-cd30-4d1a-be82-ae75f7b1cde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:07.099212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1090/2139] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:07.112612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28375f0-f12d-42c9-a82c-e1ff5b4d106e admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:07.114394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b28375f0-f12d-42c9-a82c-e1ff5b4d106e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:07.219010 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28375f0-f12d-42c9-a82c-e1ff5b4d106e admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:07.223770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f2346-bab0-4e3d-8a16-db70765746df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:07.224526 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f2346-bab0-4e3d-8a16-db70765746df service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:07.224815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f2346-bab0-4e3d-8a16-db70765746df service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:07.225109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f2346-bab0-4e3d-8a16-db70765746df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:07.234827 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f2346-bab0-4e3d-8a16-db70765746df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:07.236432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f2346-bab0-4e3d-8a16-db70765746df service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:07.247140 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1050/2140] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:31:12.052205 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e9559-cd30-4d1a-be82-ae75f7b1cde9 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:12.053587 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-345e9559-cd30-4d1a-be82-ae75f7b1cde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:12.118555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e9559-cd30-4d1a-be82-ae75f7b1cde9 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:12.124546 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217597e-5023-4e26-a0ac-dcb0de09e912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:12.125486 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217597e-5023-4e26-a0ac-dcb0de09e912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:12.125715 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217597e-5023-4e26-a0ac-dcb0de09e912 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:12.126038 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217597e-5023-4e26-a0ac-dcb0de09e912 admin admin] PATH_INFO: `/v3/users/148bc27efd504ca8997d5b55d61f33ec` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:12.127574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217597e-5023-4e26-a0ac-dcb0de09e912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=148bc27efd504ca8997d5b55d61f33ec)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:12.129493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217597e-5023-4e26-a0ac-dcb0de09e912 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:12.175119 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c217597e-5023-4e26-a0ac-dcb0de09e912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '148bc27efd504ca8997d5b55d61f33ec'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:12.175875 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c217597e-5023-4e26-a0ac-dcb0de09e912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '148bc27efd504ca8997d5b55d61f33ec'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:12.184380 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c217597e-5023-4e26-a0ac-dcb0de09e912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '148bc27efd504ca8997d5b55d61f33ec'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:12.191231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c217597e-5023-4e26-a0ac-dcb0de09e912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '148bc27efd504ca8997d5b55d61f33ec'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:12.191867 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c217597e-5023-4e26-a0ac-dcb0de09e912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '148bc27efd504ca8997d5b55d61f33ec'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:12.208771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1091/2141] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:31:12 2022] DELETE /identity/v3/users/148bc27efd504ca8997d5b55d61f33ec => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:12.213271 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb543b-2197-473a-b5e5-c6ba641b3a65 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:12.213792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb543b-2197-473a-b5e5-c6ba641b3a65 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:12.213904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb543b-2197-473a-b5e5-c6ba641b3a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:12.230887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-7dcb543b-2197-473a-b5e5-c6ba641b3a65 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:12.232151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7dcb543b-2197-473a-b5e5-c6ba641b3a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:12.290637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1051/2142] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:12.300093 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217597e-5023-4e26-a0ac-dcb0de09e912 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:12.301542 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c217597e-5023-4e26-a0ac-dcb0de09e912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:12.362315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217597e-5023-4e26-a0ac-dcb0de09e912 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:12.366467 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f22cc7-3006-4b40-81bd-1888bb87fcf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:12.367076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f22cc7-3006-4b40-81bd-1888bb87fcf4 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:12.367532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f22cc7-3006-4b40-81bd-1888bb87fcf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:12.367925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f22cc7-3006-4b40-81bd-1888bb87fcf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:12.378659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f22cc7-3006-4b40-81bd-1888bb87fcf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:12.380212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f22cc7-3006-4b40-81bd-1888bb87fcf4 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:12.391366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1092/2143] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:31:12.737381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb543b-2197-473a-b5e5-c6ba641b3a65 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:12.738811 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7dcb543b-2197-473a-b5e5-c6ba641b3a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:12.758925 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb543b-2197-473a-b5e5-c6ba641b3a65 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:12.764850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:12.765798 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:12.766151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:12.766495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] PATH_INFO: `/v3/projects/22e5f97c8b1046c3a66d910502001a8b` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:12.772084 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22e5f97c8b1046c3a66d910502001a8b)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:12.774117 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:12.824011 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22e5f97c8b1046c3a66d910502001a8b', 'request_id': 'req-d2403a16-3d1a-420a-b204-2163069e37d8'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:12.831074 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22e5f97c8b1046c3a66d910502001a8b', 'request_id': 'req-d2403a16-3d1a-420a-b204-2163069e37d8'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:12.847011 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22e5f97c8b1046c3a66d910502001a8b', 'request_id': 'req-d2403a16-3d1a-420a-b204-2163069e37d8'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:12.860419 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] The token cache is being invalidate because project 22e5f97c8b1046c3a66d910502001a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:31:12.860714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:12.862685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:12.863952 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1052/2144] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:31:12 2022] DELETE /identity/v3/projects/22e5f97c8b1046c3a66d910502001a8b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:12.867927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f22cc7-3006-4b40-81bd-1888bb87fcf4 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:12.877408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f22cc7-3006-4b40-81bd-1888bb87fcf4 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:12.934879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1057c1a-76ec-4438-b74b-73dd33c09bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:12.935847 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1057c1a-76ec-4438-b74b-73dd33c09bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:12.936093 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1057c1a-76ec-4438-b74b-73dd33c09bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:12.936369 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1057c1a-76ec-4438-b74b-73dd33c09bb8 admin admin] PATH_INFO: `/v3/users/03380e6fb869472c9f23024544e29d4b` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:12.939939 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1057c1a-76ec-4438-b74b-73dd33c09bb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03380e6fb869472c9f23024544e29d4b)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:12.941873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1057c1a-76ec-4438-b74b-73dd33c09bb8 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:12.997679 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c1057c1a-76ec-4438-b74b-73dd33c09bb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03380e6fb869472c9f23024544e29d4b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:12.998581 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c1057c1a-76ec-4438-b74b-73dd33c09bb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03380e6fb869472c9f23024544e29d4b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:13.006558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c1057c1a-76ec-4438-b74b-73dd33c09bb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03380e6fb869472c9f23024544e29d4b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:13.014504 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c1057c1a-76ec-4438-b74b-73dd33c09bb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03380e6fb869472c9f23024544e29d4b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:13.015278 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c1057c1a-76ec-4438-b74b-73dd33c09bb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03380e6fb869472c9f23024544e29d4b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:13.034730 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1093/2145] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:31:12 2022] DELETE /identity/v3/users/03380e6fb869472c9f23024544e29d4b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:13.534160 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:13.536995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:13.551151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2403a16-3d1a-420a-b204-2163069e37d8 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:13.630741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18435f-c886-4f4f-bdc7-e2b43590f182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:13.631707 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18435f-c886-4f4f-bdc7-e2b43590f182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:13.632033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18435f-c886-4f4f-bdc7-e2b43590f182 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:13.632454 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18435f-c886-4f4f-bdc7-e2b43590f182 admin admin] PATH_INFO: `/v3/projects/657bbcc599404bbeb4966588fbc46a10` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:13.634237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18435f-c886-4f4f-bdc7-e2b43590f182 admin admin] RBAC: Authorizing `identity:delete_project(project_id=657bbcc599404bbeb4966588fbc46a10)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:13.638548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18435f-c886-4f4f-bdc7-e2b43590f182 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:13.687699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1d18435f-c886-4f4f-bdc7-e2b43590f182 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '657bbcc599404bbeb4966588fbc46a10', 'request_id': 'req-1d18435f-c886-4f4f-bdc7-e2b43590f182'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:13.693459 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1d18435f-c886-4f4f-bdc7-e2b43590f182 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '657bbcc599404bbeb4966588fbc46a10', 'request_id': 'req-1d18435f-c886-4f4f-bdc7-e2b43590f182'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:13.701624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1d18435f-c886-4f4f-bdc7-e2b43590f182 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '657bbcc599404bbeb4966588fbc46a10', 'request_id': 'req-1d18435f-c886-4f4f-bdc7-e2b43590f182'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:13.714722 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1d18435f-c886-4f4f-bdc7-e2b43590f182 admin admin] The token cache is being invalidate because project 657bbcc599404bbeb4966588fbc46a10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:31:13.714722 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1d18435f-c886-4f4f-bdc7-e2b43590f182 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:13.715699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1d18435f-c886-4f4f-bdc7-e2b43590f182 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:13.721452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1053/2146] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:31:13 2022] DELETE /identity/v3/projects/657bbcc599404bbeb4966588fbc46a10 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:13.727128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2750a-61ee-483c-904b-14d68615fd88 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:13.727128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2750a-61ee-483c-904b-14d68615fd88 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:13.727128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2750a-61ee-483c-904b-14d68615fd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:13.754371 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-56f2750a-61ee-483c-904b-14d68615fd88 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:13.755641 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-56f2750a-61ee-483c-904b-14d68615fd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:13.820564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1094/2147] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:13.828936 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfc74e-58b8-4c2f-a119-8f0111b4ac10 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:13.829238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfc74e-58b8-4c2f-a119-8f0111b4ac10 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:13.829514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfc74e-58b8-4c2f-a119-8f0111b4ac10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:13.850167 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-fccfc74e-58b8-4c2f-a119-8f0111b4ac10 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:13.851410 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-fccfc74e-58b8-4c2f-a119-8f0111b4ac10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:13.860525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1054/2148] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:13.865142 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2750a-61ee-483c-904b-14d68615fd88 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:13.875685 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2750a-61ee-483c-904b-14d68615fd88 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:13.880827 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096577f-39e8-468b-82e8-b977c4a4a3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:13.881558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096577f-39e8-468b-82e8-b977c4a4a3dc admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:13.881918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096577f-39e8-468b-82e8-b977c4a4a3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:13.882263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096577f-39e8-468b-82e8-b977c4a4a3dc admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:13.883060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096577f-39e8-468b-82e8-b977c4a4a3dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:31:13.883536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096577f-39e8-468b-82e8-b977c4a4a3dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:13.887221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096577f-39e8-468b-82e8-b977c4a4a3dc admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:13.901881 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1095/2149] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:31:13.905845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfc74e-58b8-4c2f-a119-8f0111b4ac10 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:13.915394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfc74e-58b8-4c2f-a119-8f0111b4ac10 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:13.920804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab167c70-8edb-483c-b1f9-00030959a03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:13.921546 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab167c70-8edb-483c-b1f9-00030959a03a admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:13.922462 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab167c70-8edb-483c-b1f9-00030959a03a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:13.922462 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab167c70-8edb-483c-b1f9-00030959a03a admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:13.922972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab167c70-8edb-483c-b1f9-00030959a03a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:13.925442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab167c70-8edb-483c-b1f9-00030959a03a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:13.955853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1055/2150] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:31:13 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:13.963337 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096577f-39e8-468b-82e8-b977c4a4a3dc admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:13.982612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096577f-39e8-468b-82e8-b977c4a4a3dc admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:14.051048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ea1fa-a307-4134-9165-5c10671f6bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:14.052000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ea1fa-a307-4134-9165-5c10671f6bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:14.052433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ea1fa-a307-4134-9165-5c10671f6bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:14.053059 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ea1fa-a307-4134-9165-5c10671f6bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:14.054136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ea1fa-a307-4134-9165-5c10671f6bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:14.056914 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ea1fa-a307-4134-9165-5c10671f6bd0 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:14.081841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1096/2151] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:31:13 2022] POST /identity/v3/users => generated 283 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:14.093848 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab167c70-8edb-483c-b1f9-00030959a03a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:14.107464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab167c70-8edb-483c-b1f9-00030959a03a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:14.116309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c2429-3105-4307-8a71-aa7cdc56180a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:14.117000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c2429-3105-4307-8a71-aa7cdc56180a admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:14.117280 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c2429-3105-4307-8a71-aa7cdc56180a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:14.117632 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c2429-3105-4307-8a71-aa7cdc56180a admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:14.118657 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c2429-3105-4307-8a71-aa7cdc56180a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:14.120379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c2429-3105-4307-8a71-aa7cdc56180a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:14.137925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ea1fa-a307-4134-9165-5c10671f6bd0 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:14.139136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1056/2152] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:31:14.150741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ea1fa-a307-4134-9165-5c10671f6bd0 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:14.165885 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858f0d8-6f5b-461b-883a-e577c1afccac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:14.166836 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858f0d8-6f5b-461b-883a-e577c1afccac admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:14.167870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858f0d8-6f5b-461b-883a-e577c1afccac admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:14.168297 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858f0d8-6f5b-461b-883a-e577c1afccac admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:14.169207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858f0d8-6f5b-461b-883a-e577c1afccac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:14.172807 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858f0d8-6f5b-461b-883a-e577c1afccac admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:14.189874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1097/2153] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:31:14.194216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c2429-3105-4307-8a71-aa7cdc56180a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:14.205004 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c2429-3105-4307-8a71-aa7cdc56180a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:14.209829 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b40160-8379-4ab0-a2ce-329ed119fa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:14.210886 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b40160-8379-4ab0-a2ce-329ed119fa6a admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:14.211263 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b40160-8379-4ab0-a2ce-329ed119fa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:14.211600 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b40160-8379-4ab0-a2ce-329ed119fa6a admin admin] PATH_INFO: `/v3/projects/ebebdafed10043d3863f2773883d217c/users/ec4849d3e54548b692c9e2311a15acd6/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:14.232625 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b40160-8379-4ab0-a2ce-329ed119fa6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebebdafed10043d3863f2773883d217c, user_id=ec4849d3e54548b692c9e2311a15acd6, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:14.236005 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b40160-8379-4ab0-a2ce-329ed119fa6a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:14.252700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1057/2154] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:31:14 2022] PUT /identity/v3/projects/ebebdafed10043d3863f2773883d217c/users/ec4849d3e54548b692c9e2311a15acd6/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:14.270516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b72c4-d89b-40cc-86d9-d0c734773029 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:14.270516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b72c4-d89b-40cc-86d9-d0c734773029 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:14.270516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b72c4-d89b-40cc-86d9-d0c734773029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:14.297579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-e30b72c4-d89b-40cc-86d9-d0c734773029 None None] MFA Rules not processed for user `ec4849d3e54548b692c9e2311a15acd6`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:14.299431 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e30b72c4-d89b-40cc-86d9-d0c734773029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:14.366832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1098/2155] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:14.371620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b40160-8379-4ab0-a2ce-329ed119fa6a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:14.438809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b40160-8379-4ab0-a2ce-329ed119fa6a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:14.443441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02174b-ced4-4c79-a348-521e5099fefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:14.444101 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02174b-ced4-4c79-a348-521e5099fefc admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:14.444382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02174b-ced4-4c79-a348-521e5099fefc admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:14.444655 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02174b-ced4-4c79-a348-521e5099fefc admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:14.445359 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02174b-ced4-4c79-a348-521e5099fefc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:14.447252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02174b-ced4-4c79-a348-521e5099fefc admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:14.473854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1058/2156] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:31:14 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:31:14.478730 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b72c4-d89b-40cc-86d9-d0c734773029 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:14.499131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b72c4-d89b-40cc-86d9-d0c734773029 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:14.576917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969bab5-2340-4c53-8581-7eccd5ee1d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:14.577597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969bab5-2340-4c53-8581-7eccd5ee1d11 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:14.577885 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969bab5-2340-4c53-8581-7eccd5ee1d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:14.578162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969bab5-2340-4c53-8581-7eccd5ee1d11 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:14.578888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969bab5-2340-4c53-8581-7eccd5ee1d11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:14.581136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969bab5-2340-4c53-8581-7eccd5ee1d11 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:14.604722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1099/2157] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:31:14 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:14.609049 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02174b-ced4-4c79-a348-521e5099fefc admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:14.622686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02174b-ced4-4c79-a348-521e5099fefc admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:14.629576 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a5eb9-6123-41c7-9184-6853484f14be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:14.630895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a5eb9-6123-41c7-9184-6853484f14be admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:14.631281 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a5eb9-6123-41c7-9184-6853484f14be admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:14.631610 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a5eb9-6123-41c7-9184-6853484f14be admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:14.632922 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a5eb9-6123-41c7-9184-6853484f14be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:14.635357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a5eb9-6123-41c7-9184-6853484f14be admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:14.651317 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1059/2158] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:31:14.653130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969bab5-2340-4c53-8581-7eccd5ee1d11 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:14.664346 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969bab5-2340-4c53-8581-7eccd5ee1d11 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:14.669782 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60885d49-a463-4c28-a12c-80e371e5b25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:14.670743 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60885d49-a463-4c28-a12c-80e371e5b25f admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:14.671024 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60885d49-a463-4c28-a12c-80e371e5b25f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:14.671329 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60885d49-a463-4c28-a12c-80e371e5b25f admin admin] PATH_INFO: `/v3/projects/56ce2438ef1447fe99e791737204a19c/users/b59cea84df31491488328a98a297ce68/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:14.700597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60885d49-a463-4c28-a12c-80e371e5b25f admin admin] RBAC: Authorizing `identity:create_grant(project_id=56ce2438ef1447fe99e791737204a19c, user_id=b59cea84df31491488328a98a297ce68, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:14.704373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60885d49-a463-4c28-a12c-80e371e5b25f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:14.720013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1100/2159] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:31:14 2022] PUT /identity/v3/projects/56ce2438ef1447fe99e791737204a19c/users/b59cea84df31491488328a98a297ce68/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:14.723971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a5eb9-6123-41c7-9184-6853484f14be admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:14.734740 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a5eb9-6123-41c7-9184-6853484f14be admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:14.807792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082a35b-dbbb-418e-99a1-4a3a883b63e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:14.808505 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082a35b-dbbb-418e-99a1-4a3a883b63e2 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:14.808960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082a35b-dbbb-418e-99a1-4a3a883b63e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:14.809260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082a35b-dbbb-418e-99a1-4a3a883b63e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:14.809992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082a35b-dbbb-418e-99a1-4a3a883b63e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:14.811623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082a35b-dbbb-418e-99a1-4a3a883b63e2 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:14.825588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1060/2160] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:31:14.826795 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60885d49-a463-4c28-a12c-80e371e5b25f admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:14.837651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60885d49-a463-4c28-a12c-80e371e5b25f admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:14.843148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096beaf1-bfa6-4a64-9267-8831136e1c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:14.844088 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096beaf1-bfa6-4a64-9267-8831136e1c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:14.844368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096beaf1-bfa6-4a64-9267-8831136e1c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:14.844640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096beaf1-bfa6-4a64-9267-8831136e1c1d admin admin] PATH_INFO: `/v3/projects/56ce2438ef1447fe99e791737204a19c/users/b59cea84df31491488328a98a297ce68/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:14.846640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096beaf1-bfa6-4a64-9267-8831136e1c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=56ce2438ef1447fe99e791737204a19c, user_id=b59cea84df31491488328a98a297ce68, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:14.849296 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096beaf1-bfa6-4a64-9267-8831136e1c1d admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:14.859731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1101/2161] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:31:14 2022] PUT /identity/v3/projects/56ce2438ef1447fe99e791737204a19c/users/b59cea84df31491488328a98a297ce68/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:14.870778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8288b36-4884-402e-8306-f0699006a97e None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:14.871063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8288b36-4884-402e-8306-f0699006a97e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:14.871372 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8288b36-4884-402e-8306-f0699006a97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:14.897588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-d8288b36-4884-402e-8306-f0699006a97e None None] MFA Rules not processed for user `b59cea84df31491488328a98a297ce68`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:14.898792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-d8288b36-4884-402e-8306-f0699006a97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:14.981950 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1061/2162] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:14.993522 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096beaf1-bfa6-4a64-9267-8831136e1c1d admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:14.995769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-096beaf1-bfa6-4a64-9267-8831136e1c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:15.055475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096beaf1-bfa6-4a64-9267-8831136e1c1d admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:15.059782 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211df0e-b7ff-4f01-bd5c-b2e98c9d7dbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:15.060500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211df0e-b7ff-4f01-bd5c-b2e98c9d7dbd service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:15.060784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211df0e-b7ff-4f01-bd5c-b2e98c9d7dbd service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:15.061104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211df0e-b7ff-4f01-bd5c-b2e98c9d7dbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:15.070366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211df0e-b7ff-4f01-bd5c-b2e98c9d7dbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:15.071873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211df0e-b7ff-4f01-bd5c-b2e98c9d7dbd service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:15.083492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1102/2163] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:31:15.159404 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8288b36-4884-402e-8306-f0699006a97e None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:15.169386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8288b36-4884-402e-8306-f0699006a97e None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:15.173563 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b392f-1c72-4db4-9bc6-2fd5ff59fc1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:15.174215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b392f-1c72-4db4-9bc6-2fd5ff59fc1b service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:15.174560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b392f-1c72-4db4-9bc6-2fd5ff59fc1b service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:15.174766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b392f-1c72-4db4-9bc6-2fd5ff59fc1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:15.240368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b392f-1c72-4db4-9bc6-2fd5ff59fc1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:15.241803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b392f-1c72-4db4-9bc6-2fd5ff59fc1b service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:15.255308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1062/2164] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:31:19.026915 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211df0e-b7ff-4f01-bd5c-b2e98c9d7dbd service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:19.087650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211df0e-b7ff-4f01-bd5c-b2e98c9d7dbd service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:19.093742 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6e180-f9be-4532-9305-a1d4b0d82a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:19.094634 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e180-f9be-4532-9305-a1d4b0d82a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:19.094912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e180-f9be-4532-9305-a1d4b0d82a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:19.095202 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e180-f9be-4532-9305-a1d4b0d82a74 admin admin] PATH_INFO: `/v3/users/ec4849d3e54548b692c9e2311a15acd6` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:19.096408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6e180-f9be-4532-9305-a1d4b0d82a74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4849d3e54548b692c9e2311a15acd6)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:19.098307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6e180-f9be-4532-9305-a1d4b0d82a74 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:19.148224 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5db6e180-f9be-4532-9305-a1d4b0d82a74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4849d3e54548b692c9e2311a15acd6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.148923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5db6e180-f9be-4532-9305-a1d4b0d82a74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4849d3e54548b692c9e2311a15acd6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.157489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5db6e180-f9be-4532-9305-a1d4b0d82a74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4849d3e54548b692c9e2311a15acd6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.164418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5db6e180-f9be-4532-9305-a1d4b0d82a74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4849d3e54548b692c9e2311a15acd6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.165171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5db6e180-f9be-4532-9305-a1d4b0d82a74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4849d3e54548b692c9e2311a15acd6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.181859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1103/2165] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:31:19 2022] DELETE /identity/v3/users/ec4849d3e54548b692c9e2311a15acd6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:19.186545 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d6959-7101-438d-be8d-39172f3d2621 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:19.187030 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d6959-7101-438d-be8d-39172f3d2621 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:19.187146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d6959-7101-438d-be8d-39172f3d2621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:19.204591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-549d6959-7101-438d-be8d-39172f3d2621 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:19.205761 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-549d6959-7101-438d-be8d-39172f3d2621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:19.262952 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1063/2166] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:19.272961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6e180-f9be-4532-9305-a1d4b0d82a74 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:19.274618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-5db6e180-f9be-4532-9305-a1d4b0d82a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:19.332019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6e180-f9be-4532-9305-a1d4b0d82a74 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:19.335655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bc9a5-2b98-41f3-bc8d-16d45eca443f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:19.336293 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bc9a5-2b98-41f3-bc8d-16d45eca443f service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:19.336570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bc9a5-2b98-41f3-bc8d-16d45eca443f service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:19.336847 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bc9a5-2b98-41f3-bc8d-16d45eca443f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:19.347292 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bc9a5-2b98-41f3-bc8d-16d45eca443f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:19.348707 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bc9a5-2b98-41f3-bc8d-16d45eca443f service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:19.358638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1104/2167] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:31:19.627765 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d6959-7101-438d-be8d-39172f3d2621 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:19.632877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-549d6959-7101-438d-be8d-39172f3d2621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:19.646422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d6959-7101-438d-be8d-39172f3d2621 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:19.651032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:19.651952 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:19.652307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:19.652586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] PATH_INFO: `/v3/projects/ebebdafed10043d3863f2773883d217c` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:19.653783 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebebdafed10043d3863f2773883d217c)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:19.655728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:19.707648 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebebdafed10043d3863f2773883d217c', 'request_id': 'req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.712958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebebdafed10043d3863f2773883d217c', 'request_id': 'req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.727221 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebebdafed10043d3863f2773883d217c', 'request_id': 'req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.741842 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] The token cache is being invalidate because project ebebdafed10043d3863f2773883d217c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:31:19.742125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.742784 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.744332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1064/2168] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:31:19 2022] DELETE /identity/v3/projects/ebebdafed10043d3863f2773883d217c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:19.749193 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bc9a5-2b98-41f3-bc8d-16d45eca443f service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:19.759720 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bc9a5-2b98-41f3-bc8d-16d45eca443f service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:19.817333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9b923-5691-404d-b070-214a8754829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:19.818207 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9b923-5691-404d-b070-214a8754829a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:19.818484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9b923-5691-404d-b070-214a8754829a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:19.818755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9b923-5691-404d-b070-214a8754829a admin admin] PATH_INFO: `/v3/users/b59cea84df31491488328a98a297ce68` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:19.819983 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9b923-5691-404d-b070-214a8754829a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b59cea84df31491488328a98a297ce68)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:19.821978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9b923-5691-404d-b070-214a8754829a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:19.871325 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e8a9b923-5691-404d-b070-214a8754829a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b59cea84df31491488328a98a297ce68'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.871846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e8a9b923-5691-404d-b070-214a8754829a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b59cea84df31491488328a98a297ce68'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.880536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e8a9b923-5691-404d-b070-214a8754829a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b59cea84df31491488328a98a297ce68'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.887938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e8a9b923-5691-404d-b070-214a8754829a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b59cea84df31491488328a98a297ce68'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.888624 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e8a9b923-5691-404d-b070-214a8754829a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b59cea84df31491488328a98a297ce68'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:19.909761 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1105/2169] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:31:19 2022] DELETE /identity/v3/users/b59cea84df31491488328a98a297ce68 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:20.193594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:20.195056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:20.208498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812caf5-dd25-448e-b4a1-2b3ac95a8e0b admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:20.273608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:20.274470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:20.274747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:20.275018 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6 admin admin] PATH_INFO: `/v3/projects/56ce2438ef1447fe99e791737204a19c` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:20.276229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56ce2438ef1447fe99e791737204a19c)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:20.278146 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:20.327429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56ce2438ef1447fe99e791737204a19c', 'request_id': 'req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:20.334379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56ce2438ef1447fe99e791737204a19c', 'request_id': 'req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:20.343709 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56ce2438ef1447fe99e791737204a19c', 'request_id': 'req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:20.355127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6 admin admin] The token cache is being invalidate because project 56ce2438ef1447fe99e791737204a19c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:31:20.355530 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:20.356233 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e30cd92f-ca22-40e1-96cc-c27a6a97ebf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:20.358054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1065/2170] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:31:20 2022] DELETE /identity/v3/projects/56ce2438ef1447fe99e791737204a19c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:31:20.363744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74851c7-0746-4af6-b19e-9d2366640dc8 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:20.364233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74851c7-0746-4af6-b19e-9d2366640dc8 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:20.364334 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74851c7-0746-4af6-b19e-9d2366640dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:20.382328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-a74851c7-0746-4af6-b19e-9d2366640dc8 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:20.383537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a74851c7-0746-4af6-b19e-9d2366640dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:20.441618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1106/2171] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:20.450820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8829383-b3a0-4957-96fe-89288f77e378 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:20.451100 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8829383-b3a0-4957-96fe-89288f77e378 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:20.451420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8829383-b3a0-4957-96fe-89288f77e378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:20.469214 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-a8829383-b3a0-4957-96fe-89288f77e378 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:20.470385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a8829383-b3a0-4957-96fe-89288f77e378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:20.478235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1066/2172] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:20.482425 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74851c7-0746-4af6-b19e-9d2366640dc8 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:20.492012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74851c7-0746-4af6-b19e-9d2366640dc8 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:20.497238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a4db0-9111-4298-8c2d-809d7ae82629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:20.497893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a4db0-9111-4298-8c2d-809d7ae82629 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:20.498172 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a4db0-9111-4298-8c2d-809d7ae82629 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:20.498443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a4db0-9111-4298-8c2d-809d7ae82629 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:20.499159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a4db0-9111-4298-8c2d-809d7ae82629 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:31:20.499487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a4db0-9111-4298-8c2d-809d7ae82629 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:20.501069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a4db0-9111-4298-8c2d-809d7ae82629 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:20.513223 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1107/2173] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:31:20.517355 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8829383-b3a0-4957-96fe-89288f77e378 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:20.527219 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8829383-b3a0-4957-96fe-89288f77e378 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:20.532133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528328f9-2038-4cd8-b2d3-8b095cdb179c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:20.532798 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528328f9-2038-4cd8-b2d3-8b095cdb179c admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:20.533079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528328f9-2038-4cd8-b2d3-8b095cdb179c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:20.533352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528328f9-2038-4cd8-b2d3-8b095cdb179c admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:20.534037 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528328f9-2038-4cd8-b2d3-8b095cdb179c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:20.535950 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528328f9-2038-4cd8-b2d3-8b095cdb179c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:20.564531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1067/2174] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:31:20 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:20.568502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a4db0-9111-4298-8c2d-809d7ae82629 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:20.579907 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a4db0-9111-4298-8c2d-809d7ae82629 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:20.636886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a83d0-17fd-4cdb-9333-2930ed082d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:20.637553 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a83d0-17fd-4cdb-9333-2930ed082d1c admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:20.637834 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a83d0-17fd-4cdb-9333-2930ed082d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:20.638105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a83d0-17fd-4cdb-9333-2930ed082d1c admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:20.638829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a83d0-17fd-4cdb-9333-2930ed082d1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:20.641085 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a83d0-17fd-4cdb-9333-2930ed082d1c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:20.663301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1108/2175] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:31:20 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:20.667918 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528328f9-2038-4cd8-b2d3-8b095cdb179c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:20.677930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528328f9-2038-4cd8-b2d3-8b095cdb179c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:20.682464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1736b05-e079-4ebc-b9db-8f48c2bcbf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:20.683755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1736b05-e079-4ebc-b9db-8f48c2bcbf90 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:20.684035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1736b05-e079-4ebc-b9db-8f48c2bcbf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:20.684313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1736b05-e079-4ebc-b9db-8f48c2bcbf90 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:20.685034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1736b05-e079-4ebc-b9db-8f48c2bcbf90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:20.686594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1736b05-e079-4ebc-b9db-8f48c2bcbf90 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:20.696706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1068/2176] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:31:20.700808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a83d0-17fd-4cdb-9333-2930ed082d1c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:20.713626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a83d0-17fd-4cdb-9333-2930ed082d1c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:20.718753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fd585-7783-45c7-b6ea-9ccd0127601b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:20.719453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fd585-7783-45c7-b6ea-9ccd0127601b admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:20.719731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fd585-7783-45c7-b6ea-9ccd0127601b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:20.720005 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fd585-7783-45c7-b6ea-9ccd0127601b admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:20.720725 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fd585-7783-45c7-b6ea-9ccd0127601b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:20.722450 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fd585-7783-45c7-b6ea-9ccd0127601b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:20.735499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1109/2177] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:31:20.739846 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1736b05-e079-4ebc-b9db-8f48c2bcbf90 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:20.750390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1736b05-e079-4ebc-b9db-8f48c2bcbf90 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:20.755139 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5924d91-abb3-4e28-8cfb-2d79da6526d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:20.756127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5924d91-abb3-4e28-8cfb-2d79da6526d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:20.756406 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5924d91-abb3-4e28-8cfb-2d79da6526d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:20.756675 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5924d91-abb3-4e28-8cfb-2d79da6526d5 admin admin] PATH_INFO: `/v3/projects/0e619429cfde4b5ca217dba7b8afc53a/users/535b60a91b1744a3b796b9bb3488813a/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:20.773008 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5924d91-abb3-4e28-8cfb-2d79da6526d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e619429cfde4b5ca217dba7b8afc53a, user_id=535b60a91b1744a3b796b9bb3488813a, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:20.775706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5924d91-abb3-4e28-8cfb-2d79da6526d5 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:20.788054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1069/2178] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:31:20 2022] PUT /identity/v3/projects/0e619429cfde4b5ca217dba7b8afc53a/users/535b60a91b1744a3b796b9bb3488813a/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:20.798907 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112de1f-8f97-45dc-97c1-cbda7fdf59b1 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:20.800107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112de1f-8f97-45dc-97c1-cbda7fdf59b1 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:20.800107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112de1f-8f97-45dc-97c1-cbda7fdf59b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:20.829765 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2112de1f-8f97-45dc-97c1-cbda7fdf59b1 None None] MFA Rules not processed for user `535b60a91b1744a3b796b9bb3488813a`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:20.831030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2112de1f-8f97-45dc-97c1-cbda7fdf59b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:20.894727 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1110/2179] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:20.899113 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5924d91-abb3-4e28-8cfb-2d79da6526d5 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:20.962268 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5924d91-abb3-4e28-8cfb-2d79da6526d5 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:20.967869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4599f2-005b-4ea3-981c-7b3aeb5ed84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:20.968527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4599f2-005b-4ea3-981c-7b3aeb5ed84f admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:20.968807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4599f2-005b-4ea3-981c-7b3aeb5ed84f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:20.969082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4599f2-005b-4ea3-981c-7b3aeb5ed84f admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:20.969758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4599f2-005b-4ea3-981c-7b3aeb5ed84f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:20.971665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4599f2-005b-4ea3-981c-7b3aeb5ed84f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:21.001957 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1070/2180] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:31:20 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:21.005803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112de1f-8f97-45dc-97c1-cbda7fdf59b1 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:21.016001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112de1f-8f97-45dc-97c1-cbda7fdf59b1 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:21.079135 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2cd270-09b0-4ece-b17d-07fe4f178066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:21.079858 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cd270-09b0-4ece-b17d-07fe4f178066 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:21.080142 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cd270-09b0-4ece-b17d-07fe4f178066 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:21.080415 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cd270-09b0-4ece-b17d-07fe4f178066 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:21.081198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2cd270-09b0-4ece-b17d-07fe4f178066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:21.083308 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2cd270-09b0-4ece-b17d-07fe4f178066 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:21.109265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1111/2181] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:31:21 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:21.112932 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4599f2-005b-4ea3-981c-7b3aeb5ed84f admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:21.124071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4599f2-005b-4ea3-981c-7b3aeb5ed84f admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:21.129423 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9634ec-8d92-47bd-912e-063e1615d461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:21.130112 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9634ec-8d92-47bd-912e-063e1615d461 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:21.130398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9634ec-8d92-47bd-912e-063e1615d461 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:21.131679 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9634ec-8d92-47bd-912e-063e1615d461 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:21.132119 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9634ec-8d92-47bd-912e-063e1615d461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:21.133705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9634ec-8d92-47bd-912e-063e1615d461 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:21.145340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1071/2182] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:31:21.149432 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2cd270-09b0-4ece-b17d-07fe4f178066 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:21.160283 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2cd270-09b0-4ece-b17d-07fe4f178066 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:21.164919 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9a017-3b3f-44a8-8059-ebf410fdd6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:21.165566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9a017-3b3f-44a8-8059-ebf410fdd6f6 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:21.165848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9a017-3b3f-44a8-8059-ebf410fdd6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:21.166123 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9a017-3b3f-44a8-8059-ebf410fdd6f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:21.166875 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9a017-3b3f-44a8-8059-ebf410fdd6f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:21.169115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9a017-3b3f-44a8-8059-ebf410fdd6f6 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:21.178892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1112/2183] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:31:21.182683 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9634ec-8d92-47bd-912e-063e1615d461 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:21.192363 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9634ec-8d92-47bd-912e-063e1615d461 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:21.196841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f5fb6-faf0-452f-bee8-43a0bab5a92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:21.197729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f5fb6-faf0-452f-bee8-43a0bab5a92c admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:21.198008 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f5fb6-faf0-452f-bee8-43a0bab5a92c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:21.198319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f5fb6-faf0-452f-bee8-43a0bab5a92c admin admin] PATH_INFO: `/v3/projects/b9b6836f0f8143b9932cd123716acad5/users/663e70c7affb4fec84419355dd84555b/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:21.212003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f5fb6-faf0-452f-bee8-43a0bab5a92c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9b6836f0f8143b9932cd123716acad5, user_id=663e70c7affb4fec84419355dd84555b, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:21.215171 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f5fb6-faf0-452f-bee8-43a0bab5a92c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:21.227467 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1072/2184] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:31:21 2022] PUT /identity/v3/projects/b9b6836f0f8143b9932cd123716acad5/users/663e70c7affb4fec84419355dd84555b/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:31:21.239337 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b5e32-f514-4a04-a1fa-ec816472506b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:21.239654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b5e32-f514-4a04-a1fa-ec816472506b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:21.239933 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b5e32-f514-4a04-a1fa-ec816472506b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:21.266580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9c6b5e32-f514-4a04-a1fa-ec816472506b None None] MFA Rules not processed for user `663e70c7affb4fec84419355dd84555b`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:21.267862 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9c6b5e32-f514-4a04-a1fa-ec816472506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:21.331648 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1113/2185] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:21.336088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f5fb6-faf0-452f-bee8-43a0bab5a92c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:21.399909 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f5fb6-faf0-452f-bee8-43a0bab5a92c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:21.405362 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e4d24-9d65-4cef-987a-f9bb1e80f0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:21.406041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e4d24-9d65-4cef-987a-f9bb1e80f0ad admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:21.406326 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e4d24-9d65-4cef-987a-f9bb1e80f0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:21.406605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e4d24-9d65-4cef-987a-f9bb1e80f0ad admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:21.407332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e4d24-9d65-4cef-987a-f9bb1e80f0ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:21.409226 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e4d24-9d65-4cef-987a-f9bb1e80f0ad admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:21.441101 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1073/2186] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:31:21 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:31:21.442566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b5e32-f514-4a04-a1fa-ec816472506b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:21.459063 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b5e32-f514-4a04-a1fa-ec816472506b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:21.522960 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f55ba9-deab-4edd-88e3-732f4423b205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:21.523677 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f55ba9-deab-4edd-88e3-732f4423b205 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:21.523959 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f55ba9-deab-4edd-88e3-732f4423b205 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:21.524233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f55ba9-deab-4edd-88e3-732f4423b205 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:21.524944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f55ba9-deab-4edd-88e3-732f4423b205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:21.527725 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f55ba9-deab-4edd-88e3-732f4423b205 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:21.548187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1114/2187] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:31:21 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:21.552393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e4d24-9d65-4cef-987a-f9bb1e80f0ad admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:21.563026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e4d24-9d65-4cef-987a-f9bb1e80f0ad admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:21.568649 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecf852-bc9c-48ad-947c-776400216e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:21.569446 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecf852-bc9c-48ad-947c-776400216e09 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:21.569728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecf852-bc9c-48ad-947c-776400216e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:21.570002 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecf852-bc9c-48ad-947c-776400216e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:21.570737 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecf852-bc9c-48ad-947c-776400216e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:21.572383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecf852-bc9c-48ad-947c-776400216e09 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:21.583800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1074/2188] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:31:21.587872 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f55ba9-deab-4edd-88e3-732f4423b205 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:21.598011 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f55ba9-deab-4edd-88e3-732f4423b205 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:21.602593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14f08b-aa2c-4a39-a0bd-654b133d1165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:21.603493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14f08b-aa2c-4a39-a0bd-654b133d1165 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:21.603771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14f08b-aa2c-4a39-a0bd-654b133d1165 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:21.604040 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14f08b-aa2c-4a39-a0bd-654b133d1165 admin admin] PATH_INFO: `/v3/projects/4a6779260d4b45b1b07240cbd7ad05c0/users/92549baed11d4774a5d44e663fd24499/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:21.618691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14f08b-aa2c-4a39-a0bd-654b133d1165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a6779260d4b45b1b07240cbd7ad05c0, user_id=92549baed11d4774a5d44e663fd24499, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:21.621466 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14f08b-aa2c-4a39-a0bd-654b133d1165 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:21.636485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1115/2189] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:31:21 2022] PUT /identity/v3/projects/4a6779260d4b45b1b07240cbd7ad05c0/users/92549baed11d4774a5d44e663fd24499/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:21.640307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecf852-bc9c-48ad-947c-776400216e09 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:21.650959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecf852-bc9c-48ad-947c-776400216e09 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:21.707330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283921b-1bcc-465d-9703-accde78d9dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:21.708027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283921b-1bcc-465d-9703-accde78d9dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:21.708308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283921b-1bcc-465d-9703-accde78d9dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:21.708582 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283921b-1bcc-465d-9703-accde78d9dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:21.709306 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283921b-1bcc-465d-9703-accde78d9dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:21.710912 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283921b-1bcc-465d-9703-accde78d9dc9 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:21.721067 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1075/2190] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:31:21.725249 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14f08b-aa2c-4a39-a0bd-654b133d1165 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:21.736178 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14f08b-aa2c-4a39-a0bd-654b133d1165 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:21.741324 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4a35e-ed7a-47d5-847a-68ba62aafd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:21.742209 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4a35e-ed7a-47d5-847a-68ba62aafd95 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:21.742488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4a35e-ed7a-47d5-847a-68ba62aafd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:21.742759 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4a35e-ed7a-47d5-847a-68ba62aafd95 admin admin] PATH_INFO: `/v3/projects/4a6779260d4b45b1b07240cbd7ad05c0/users/92549baed11d4774a5d44e663fd24499/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:21.744743 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4a35e-ed7a-47d5-847a-68ba62aafd95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a6779260d4b45b1b07240cbd7ad05c0, user_id=92549baed11d4774a5d44e663fd24499, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:21.747730 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4a35e-ed7a-47d5-847a-68ba62aafd95 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:21.759462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1116/2191] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:31:21 2022] PUT /identity/v3/projects/4a6779260d4b45b1b07240cbd7ad05c0/users/92549baed11d4774a5d44e663fd24499/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:31:21.770356 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61fef8-f9d2-44eb-9fed-825911157b00 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:21.770644 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61fef8-f9d2-44eb-9fed-825911157b00 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:21.770924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61fef8-f9d2-44eb-9fed-825911157b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:21.798350 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-4f61fef8-f9d2-44eb-9fed-825911157b00 None None] MFA Rules not processed for user `92549baed11d4774a5d44e663fd24499`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:21.799726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-4f61fef8-f9d2-44eb-9fed-825911157b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:21.868567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1076/2192] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:21.880427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4a35e-ed7a-47d5-847a-68ba62aafd95 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:21.882137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-71e4a35e-ed7a-47d5-847a-68ba62aafd95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:21.938975 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4a35e-ed7a-47d5-847a-68ba62aafd95 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:21.942534 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4889293-6a62-411f-8739-0025af241cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:21.943168 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4889293-6a62-411f-8739-0025af241cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:21.943470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4889293-6a62-411f-8739-0025af241cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:21.943741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4889293-6a62-411f-8739-0025af241cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:21.999646 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4889293-6a62-411f-8739-0025af241cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:22.001071 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4889293-6a62-411f-8739-0025af241cf2 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:22.010023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1117/2193] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:31:23.504803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61fef8-f9d2-44eb-9fed-825911157b00 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:23.518439 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61fef8-f9d2-44eb-9fed-825911157b00 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:23.522662 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93283e9b-a06f-49b7-a7af-004abd897511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:23.523388 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93283e9b-a06f-49b7-a7af-004abd897511 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:23.523679 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93283e9b-a06f-49b7-a7af-004abd897511 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:23.523955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93283e9b-a06f-49b7-a7af-004abd897511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:23.582837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93283e9b-a06f-49b7-a7af-004abd897511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:23.584209 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93283e9b-a06f-49b7-a7af-004abd897511 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:23.593094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1077/2194] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:31:24.164956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4889293-6a62-411f-8739-0025af241cf2 service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:24.173959 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4889293-6a62-411f-8739-0025af241cf2 service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:24.177790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b00b8-d069-4cd8-b333-c2601d9358c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:24.178429 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b00b8-d069-4cd8-b333-c2601d9358c4 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:24.178710 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b00b8-d069-4cd8-b333-c2601d9358c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:24.178987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b00b8-d069-4cd8-b333-c2601d9358c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:24.187971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b00b8-d069-4cd8-b333-c2601d9358c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:24.189634 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b00b8-d069-4cd8-b333-c2601d9358c4 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:24.201874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1118/2195] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:31:28.044069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93283e9b-a06f-49b7-a7af-004abd897511 service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:28.124078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93283e9b-a06f-49b7-a7af-004abd897511 service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:28.129007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4aed8-35bf-4d36-9332-5a5d3b2c7334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:28.129908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4aed8-35bf-4d36-9332-5a5d3b2c7334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:28.130195 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4aed8-35bf-4d36-9332-5a5d3b2c7334 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:28.130473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4aed8-35bf-4d36-9332-5a5d3b2c7334 admin admin] PATH_INFO: `/v3/users/535b60a91b1744a3b796b9bb3488813a` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:28.131738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4aed8-35bf-4d36-9332-5a5d3b2c7334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=535b60a91b1744a3b796b9bb3488813a)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:28.133706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4aed8-35bf-4d36-9332-5a5d3b2c7334 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:28.191940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-36f4aed8-35bf-4d36-9332-5a5d3b2c7334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '535b60a91b1744a3b796b9bb3488813a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.199195 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-36f4aed8-35bf-4d36-9332-5a5d3b2c7334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '535b60a91b1744a3b796b9bb3488813a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.199878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-36f4aed8-35bf-4d36-9332-5a5d3b2c7334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '535b60a91b1744a3b796b9bb3488813a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.209941 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-36f4aed8-35bf-4d36-9332-5a5d3b2c7334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '535b60a91b1744a3b796b9bb3488813a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.210737 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-36f4aed8-35bf-4d36-9332-5a5d3b2c7334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '535b60a91b1744a3b796b9bb3488813a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.235273 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1078/2196] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:31:28 2022] DELETE /identity/v3/users/535b60a91b1744a3b796b9bb3488813a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:28.239722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12353d09-a216-4482-bbde-f7a3a88b91a6 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:28.240244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12353d09-a216-4482-bbde-f7a3a88b91a6 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:28.240379 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12353d09-a216-4482-bbde-f7a3a88b91a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:28.258105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-12353d09-a216-4482-bbde-f7a3a88b91a6 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:28.259415 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-12353d09-a216-4482-bbde-f7a3a88b91a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:28.319352 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1119/2197] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:28.329006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4aed8-35bf-4d36-9332-5a5d3b2c7334 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:28.330398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-36f4aed8-35bf-4d36-9332-5a5d3b2c7334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:28.400463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4aed8-35bf-4d36-9332-5a5d3b2c7334 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:28.405169 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bccaa-35e7-40e5-bf86-5ceb97dc6061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:28.405835 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bccaa-35e7-40e5-bf86-5ceb97dc6061 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:28.406421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bccaa-35e7-40e5-bf86-5ceb97dc6061 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:28.406421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bccaa-35e7-40e5-bf86-5ceb97dc6061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:28.415981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bccaa-35e7-40e5-bf86-5ceb97dc6061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:28.417960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bccaa-35e7-40e5-bf86-5ceb97dc6061 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:28.427791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1079/2198] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:31:28.701752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12353d09-a216-4482-bbde-f7a3a88b91a6 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:28.703267 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-12353d09-a216-4482-bbde-f7a3a88b91a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:28.717720 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12353d09-a216-4482-bbde-f7a3a88b91a6 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:28.722303 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:28.723212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:28.723495 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:28.723769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] PATH_INFO: `/v3/projects/0e619429cfde4b5ca217dba7b8afc53a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:28.724929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e619429cfde4b5ca217dba7b8afc53a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:28.726809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:28.774840 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e619429cfde4b5ca217dba7b8afc53a', 'request_id': 'req-8ff135c9-d168-4725-9f7a-65c6f0abb647'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.780484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e619429cfde4b5ca217dba7b8afc53a', 'request_id': 'req-8ff135c9-d168-4725-9f7a-65c6f0abb647'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.787447 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e619429cfde4b5ca217dba7b8afc53a', 'request_id': 'req-8ff135c9-d168-4725-9f7a-65c6f0abb647'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.800381 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] The token cache is being invalidate because project 0e619429cfde4b5ca217dba7b8afc53a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:31:28.800668 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.801455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.802984 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1120/2199] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:31:28 2022] DELETE /identity/v3/projects/0e619429cfde4b5ca217dba7b8afc53a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:28.806925 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bccaa-35e7-40e5-bf86-5ceb97dc6061 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:28.816524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bccaa-35e7-40e5-bf86-5ceb97dc6061 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:28.875098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46b58d-df20-4458-b869-e6b7a99617a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:28.875924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46b58d-df20-4458-b869-e6b7a99617a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:28.876494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46b58d-df20-4458-b869-e6b7a99617a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:28.876775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46b58d-df20-4458-b869-e6b7a99617a4 admin admin] PATH_INFO: `/v3/users/663e70c7affb4fec84419355dd84555b` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:28.877953 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d46b58d-df20-4458-b869-e6b7a99617a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=663e70c7affb4fec84419355dd84555b)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:28.879910 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d46b58d-df20-4458-b869-e6b7a99617a4 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:28.926322 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9d46b58d-df20-4458-b869-e6b7a99617a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '663e70c7affb4fec84419355dd84555b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.933350 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9d46b58d-df20-4458-b869-e6b7a99617a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '663e70c7affb4fec84419355dd84555b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.934132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9d46b58d-df20-4458-b869-e6b7a99617a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '663e70c7affb4fec84419355dd84555b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.943229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9d46b58d-df20-4458-b869-e6b7a99617a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '663e70c7affb4fec84419355dd84555b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.943941 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9d46b58d-df20-4458-b869-e6b7a99617a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '663e70c7affb4fec84419355dd84555b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:28.965044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1080/2200] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:31:28 2022] DELETE /identity/v3/users/663e70c7affb4fec84419355dd84555b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:29.238366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:29.239853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:29.262481 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff135c9-d168-4725-9f7a-65c6f0abb647 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:29.325265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:29.326057 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:29.326333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:29.326603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] PATH_INFO: `/v3/projects/b9b6836f0f8143b9932cd123716acad5` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:29.327808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9b6836f0f8143b9932cd123716acad5)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:29.329700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:29.378806 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9b6836f0f8143b9932cd123716acad5', 'request_id': 'req-36b641c8-bcce-430d-bde3-923d3df98ae9'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:29.392049 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9b6836f0f8143b9932cd123716acad5', 'request_id': 'req-36b641c8-bcce-430d-bde3-923d3df98ae9'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:29.399733 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9b6836f0f8143b9932cd123716acad5', 'request_id': 'req-36b641c8-bcce-430d-bde3-923d3df98ae9'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:29.413913 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] The token cache is being invalidate because project b9b6836f0f8143b9932cd123716acad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:31:29.414197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:29.414898 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:29.417163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1121/2201] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:31:29 2022] DELETE /identity/v3/projects/b9b6836f0f8143b9932cd123716acad5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:29.421470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46b58d-df20-4458-b869-e6b7a99617a4 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:29.422971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9d46b58d-df20-4458-b869-e6b7a99617a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:29.436622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46b58d-df20-4458-b869-e6b7a99617a4 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:29.495206 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4964596-22ca-4a47-87b2-e39928d1050f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:29.496091 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4964596-22ca-4a47-87b2-e39928d1050f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:29.496371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4964596-22ca-4a47-87b2-e39928d1050f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:29.496644 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4964596-22ca-4a47-87b2-e39928d1050f admin admin] PATH_INFO: `/v3/users/92549baed11d4774a5d44e663fd24499` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:29.497832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4964596-22ca-4a47-87b2-e39928d1050f admin admin] RBAC: Authorizing `identity:delete_user(user_id=92549baed11d4774a5d44e663fd24499)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:29.499777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4964596-22ca-4a47-87b2-e39928d1050f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:29.545871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a4964596-22ca-4a47-87b2-e39928d1050f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92549baed11d4774a5d44e663fd24499'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:29.553243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a4964596-22ca-4a47-87b2-e39928d1050f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92549baed11d4774a5d44e663fd24499'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:29.554003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a4964596-22ca-4a47-87b2-e39928d1050f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92549baed11d4774a5d44e663fd24499'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:29.563257 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a4964596-22ca-4a47-87b2-e39928d1050f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92549baed11d4774a5d44e663fd24499'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:29.563870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-a4964596-22ca-4a47-87b2-e39928d1050f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92549baed11d4774a5d44e663fd24499'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:29.582370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1081/2202] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:31:29 2022] DELETE /identity/v3/users/92549baed11d4774a5d44e663fd24499 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:31:29.855502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:29.857913 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:29.875305 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b641c8-bcce-430d-bde3-923d3df98ae9 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:29.936750 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:29.937658 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:29.937942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:29.938310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c admin admin] PATH_INFO: `/v3/projects/4a6779260d4b45b1b07240cbd7ad05c0` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:29.939442 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a6779260d4b45b1b07240cbd7ad05c0)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:29.941540 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:29.992082 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a6779260d4b45b1b07240cbd7ad05c0', 'request_id': 'req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:29.998546 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a6779260d4b45b1b07240cbd7ad05c0', 'request_id': 'req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:30.014228 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a6779260d4b45b1b07240cbd7ad05c0', 'request_id': 'req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:30.024253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c admin admin] The token cache is being invalidate because project 4a6779260d4b45b1b07240cbd7ad05c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:31:30.024547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:30.025208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ee66e2c9-359c-4a7f-93cc-2e457aa3263c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:30.027397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1122/2203] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:31:29 2022] DELETE /identity/v3/projects/4a6779260d4b45b1b07240cbd7ad05c0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:30.034500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631cdc5-7829-46e9-9f41-bd230bac13f0 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:30.034791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631cdc5-7829-46e9-9f41-bd230bac13f0 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:30.035066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631cdc5-7829-46e9-9f41-bd230bac13f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:30.052836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-1631cdc5-7829-46e9-9f41-bd230bac13f0 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:30.054009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1631cdc5-7829-46e9-9f41-bd230bac13f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:30.115552 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1082/2204] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:30.124619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905db9d-9c79-49e5-9fe3-cc47cb982106 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:30.124935 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905db9d-9c79-49e5-9fe3-cc47cb982106 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:30.125232 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905db9d-9c79-49e5-9fe3-cc47cb982106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:30.144547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-0905db9d-9c79-49e5-9fe3-cc47cb982106 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:30.145744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-0905db9d-9c79-49e5-9fe3-cc47cb982106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:30.154152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1123/2205] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:30.158529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631cdc5-7829-46e9-9f41-bd230bac13f0 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:30.169558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631cdc5-7829-46e9-9f41-bd230bac13f0 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:30.174308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52413d6-8e00-4fe6-9091-11686defd10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:30.174969 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52413d6-8e00-4fe6-9091-11686defd10c admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:30.175286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52413d6-8e00-4fe6-9091-11686defd10c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:30.175564 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52413d6-8e00-4fe6-9091-11686defd10c admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:30.176557 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52413d6-8e00-4fe6-9091-11686defd10c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:31:30.176868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52413d6-8e00-4fe6-9091-11686defd10c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:30.178843 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52413d6-8e00-4fe6-9091-11686defd10c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:30.191214 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1083/2206] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:31:30.195252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905db9d-9c79-49e5-9fe3-cc47cb982106 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:30.204274 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905db9d-9c79-49e5-9fe3-cc47cb982106 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:30.209884 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f3c01-a13d-4c5e-8f92-4a40a7c44c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:30.210554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f3c01-a13d-4c5e-8f92-4a40a7c44c4b admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:30.210837 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f3c01-a13d-4c5e-8f92-4a40a7c44c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:30.211113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f3c01-a13d-4c5e-8f92-4a40a7c44c4b admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:30.211844 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f3c01-a13d-4c5e-8f92-4a40a7c44c4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:30.214253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f3c01-a13d-4c5e-8f92-4a40a7c44c4b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:30.241478 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1124/2207] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:31:30 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:30.245741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52413d6-8e00-4fe6-9091-11686defd10c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:30.256072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52413d6-8e00-4fe6-9091-11686defd10c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:30.312641 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e791317-660d-48ad-9072-06e64225b403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:30.313319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e791317-660d-48ad-9072-06e64225b403 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:30.313597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e791317-660d-48ad-9072-06e64225b403 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:30.313868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e791317-660d-48ad-9072-06e64225b403 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:30.314595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e791317-660d-48ad-9072-06e64225b403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:30.316522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e791317-660d-48ad-9072-06e64225b403 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:30.339418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1084/2208] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:31:30 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:31:30.343593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f3c01-a13d-4c5e-8f92-4a40a7c44c4b admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:30.354084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f3c01-a13d-4c5e-8f92-4a40a7c44c4b admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:30.359100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e99d1-f981-4eac-829a-a8d6ec6d25c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:30.359926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e99d1-f981-4eac-829a-a8d6ec6d25c4 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:30.360214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e99d1-f981-4eac-829a-a8d6ec6d25c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:30.360491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e99d1-f981-4eac-829a-a8d6ec6d25c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:30.361214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e99d1-f981-4eac-829a-a8d6ec6d25c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:30.362838 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e99d1-f981-4eac-829a-a8d6ec6d25c4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:30.377115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1125/2209] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:31:30.378189 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e791317-660d-48ad-9072-06e64225b403 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:30.389074 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e791317-660d-48ad-9072-06e64225b403 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:30.393801 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcc8a6-21ce-4ad7-bca9-9460e61435d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:30.394479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcc8a6-21ce-4ad7-bca9-9460e61435d1 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:30.394758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcc8a6-21ce-4ad7-bca9-9460e61435d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:30.395029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcc8a6-21ce-4ad7-bca9-9460e61435d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:30.395780 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fcc8a6-21ce-4ad7-bca9-9460e61435d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:30.397353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fcc8a6-21ce-4ad7-bca9-9460e61435d1 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:30.412170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1085/2210] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:31:30.413558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e99d1-f981-4eac-829a-a8d6ec6d25c4 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:30.426248 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e99d1-f981-4eac-829a-a8d6ec6d25c4 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:30.430970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bc9e8-4b92-44b9-9cb5-1fd62f6629f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:30.431927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bc9e8-4b92-44b9-9cb5-1fd62f6629f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:30.432203 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bc9e8-4b92-44b9-9cb5-1fd62f6629f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:30.432478 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bc9e8-4b92-44b9-9cb5-1fd62f6629f3 admin admin] PATH_INFO: `/v3/projects/ac5d0c2f104f433e817ee1feded14e1f/users/123f6c5cd54f4708ae4ac1cbb1945264/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:30.448809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bc9e8-4b92-44b9-9cb5-1fd62f6629f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac5d0c2f104f433e817ee1feded14e1f, user_id=123f6c5cd54f4708ae4ac1cbb1945264, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:30.451470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bc9e8-4b92-44b9-9cb5-1fd62f6629f3 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:30.467863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1126/2211] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:31:30 2022] PUT /identity/v3/projects/ac5d0c2f104f433e817ee1feded14e1f/users/123f6c5cd54f4708ae4ac1cbb1945264/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:30.478498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93f2c3-5d95-4076-ad1d-c1a88d001ad4 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:30.479048 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93f2c3-5d95-4076-ad1d-c1a88d001ad4 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:30.479324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93f2c3-5d95-4076-ad1d-c1a88d001ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:30.505581 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-1b93f2c3-5d95-4076-ad1d-c1a88d001ad4 None None] MFA Rules not processed for user `123f6c5cd54f4708ae4ac1cbb1945264`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:30.506859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1b93f2c3-5d95-4076-ad1d-c1a88d001ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:30.573212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1086/2212] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:30.585003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bc9e8-4b92-44b9-9cb5-1fd62f6629f3 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:30.586676 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-dc2bc9e8-4b92-44b9-9cb5-1fd62f6629f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:30.645739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bc9e8-4b92-44b9-9cb5-1fd62f6629f3 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:30.649373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49097ac4-3966-47c6-bb02-adae35084b6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:30.650016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49097ac4-3966-47c6-bb02-adae35084b6c service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:30.650298 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49097ac4-3966-47c6-bb02-adae35084b6c service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:30.650574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49097ac4-3966-47c6-bb02-adae35084b6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:30.659395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49097ac4-3966-47c6-bb02-adae35084b6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:30.660808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49097ac4-3966-47c6-bb02-adae35084b6c service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:30.670047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1127/2213] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:31:38.754724 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93f2c3-5d95-4076-ad1d-c1a88d001ad4 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:38.816809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93f2c3-5d95-4076-ad1d-c1a88d001ad4 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:38.821315 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca774caa-f64c-4461-bac5-a19661e3384a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:38.822189 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca774caa-f64c-4461-bac5-a19661e3384a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:38.822472 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca774caa-f64c-4461-bac5-a19661e3384a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:38.823287 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca774caa-f64c-4461-bac5-a19661e3384a admin admin] PATH_INFO: `/v3/users/123f6c5cd54f4708ae4ac1cbb1945264` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:38.824463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca774caa-f64c-4461-bac5-a19661e3384a admin admin] RBAC: Authorizing `identity:delete_user(user_id=123f6c5cd54f4708ae4ac1cbb1945264)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:38.826356 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca774caa-f64c-4461-bac5-a19661e3384a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:38.869109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ca774caa-f64c-4461-bac5-a19661e3384a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '123f6c5cd54f4708ae4ac1cbb1945264'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:38.875678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ca774caa-f64c-4461-bac5-a19661e3384a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '123f6c5cd54f4708ae4ac1cbb1945264'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:38.876525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ca774caa-f64c-4461-bac5-a19661e3384a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '123f6c5cd54f4708ae4ac1cbb1945264'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:38.883461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ca774caa-f64c-4461-bac5-a19661e3384a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '123f6c5cd54f4708ae4ac1cbb1945264'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:38.884076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ca774caa-f64c-4461-bac5-a19661e3384a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '123f6c5cd54f4708ae4ac1cbb1945264'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:38.902346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1087/2214] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:31:38 2022] DELETE /identity/v3/users/123f6c5cd54f4708ae4ac1cbb1945264 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:38.907106 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6466e-2a0d-4298-a9c9-da4ac59fed33 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:38.907586 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6466e-2a0d-4298-a9c9-da4ac59fed33 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:38.907743 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6466e-2a0d-4298-a9c9-da4ac59fed33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:38.924504 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-56e6466e-2a0d-4298-a9c9-da4ac59fed33 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:38.925710 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-56e6466e-2a0d-4298-a9c9-da4ac59fed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:38.984472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1128/2215] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:31:38.996056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca774caa-f64c-4461-bac5-a19661e3384a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:38.997489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ca774caa-f64c-4461-bac5-a19661e3384a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:39.054127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca774caa-f64c-4461-bac5-a19661e3384a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:39.058316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359313c2-f956-4a50-ad1a-a94fb7851b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:39.058966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359313c2-f956-4a50-ad1a-a94fb7851b2b service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:39.059306 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359313c2-f956-4a50-ad1a-a94fb7851b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:39.059590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359313c2-f956-4a50-ad1a-a94fb7851b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:39.068371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359313c2-f956-4a50-ad1a-a94fb7851b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:39.069791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359313c2-f956-4a50-ad1a-a94fb7851b2b service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:39.079656 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1088/2216] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:31:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:31:39.350072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6466e-2a0d-4298-a9c9-da4ac59fed33 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:31:39.351548 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-56e6466e-2a0d-4298-a9c9-da4ac59fed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:39.369193 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6466e-2a0d-4298-a9c9-da4ac59fed33 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:31:39.378641 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:31:39.379627 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:39.379911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:39.380186 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] PATH_INFO: `/v3/projects/ac5d0c2f104f433e817ee1feded14e1f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:39.381460 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac5d0c2f104f433e817ee1feded14e1f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:31:39.383438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:31:39.458818 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac5d0c2f104f433e817ee1feded14e1f', 'request_id': 'req-9ad92c84-2991-48fd-a847-f5c28248504a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:39.464045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac5d0c2f104f433e817ee1feded14e1f', 'request_id': 'req-9ad92c84-2991-48fd-a847-f5c28248504a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:39.471164 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac5d0c2f104f433e817ee1feded14e1f', 'request_id': 'req-9ad92c84-2991-48fd-a847-f5c28248504a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:39.482345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] The token cache is being invalidate because project ac5d0c2f104f433e817ee1feded14e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:31:39.482630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:39.483280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:31:39.484827 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1129/2217] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:31:39 2022] DELETE /identity/v3/projects/ac5d0c2f104f433e817ee1feded14e1f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:31:46.623627 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cacb06-a9f6-43f2-b4f3-5d7ce4850aaa None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:31:46.624138 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cacb06-a9f6-43f2-b4f3-5d7ce4850aaa None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:31:46.624231 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cacb06-a9f6-43f2-b4f3-5d7ce4850aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:31:46.642296 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-75cacb06-a9f6-43f2-b4f3-5d7ce4850aaa None None] MFA Rules not processed for user `f99208187e4e4d2eb9f52107ae8f50f2`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:31:46.643525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-75cacb06-a9f6-43f2-b4f3-5d7ce4850aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:31:46.696239 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1089/2218] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:31:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:02.244635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:02.246357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:02.376221 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cacb06-a9f6-43f2-b4f3-5d7ce4850aaa None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:02.377684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-75cacb06-a9f6-43f2-b4f3-5d7ce4850aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:02.391378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad92c84-2991-48fd-a847-f5c28248504a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:02.396047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f453d-d27d-4725-b371-9f77bb45d9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:02.396920 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f453d-d27d-4725-b371-9f77bb45d9dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:02.397201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f453d-d27d-4725-b371-9f77bb45d9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:02.397473 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f453d-d27d-4725-b371-9f77bb45d9dc admin admin] PATH_INFO: `/v3/users/abae69a93a014143ab0cebfd1a7be830` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:02.398665 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f453d-d27d-4725-b371-9f77bb45d9dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=abae69a93a014143ab0cebfd1a7be830)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:02.400739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f453d-d27d-4725-b371-9f77bb45d9dc admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:02.422439 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cacb06-a9f6-43f2-b4f3-5d7ce4850aaa None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:02.428674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29953f63-f25b-4197-87e1-652ac75a940d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:02.429609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29953f63-f25b-4197-87e1-652ac75a940d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:02.429895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29953f63-f25b-4197-87e1-652ac75a940d admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:02.430170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29953f63-f25b-4197-87e1-652ac75a940d admin admin] PATH_INFO: `/v3/users/96cdbb1be17c4352ba93c31238c8b96b` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:02.431725 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29953f63-f25b-4197-87e1-652ac75a940d admin admin] RBAC: Authorizing `identity:delete_user(user_id=96cdbb1be17c4352ba93c31238c8b96b)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:02.434051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29953f63-f25b-4197-87e1-652ac75a940d admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:02.453069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3a7f453d-d27d-4725-b371-9f77bb45d9dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abae69a93a014143ab0cebfd1a7be830'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:02.455116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3a7f453d-d27d-4725-b371-9f77bb45d9dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abae69a93a014143ab0cebfd1a7be830'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:02.477196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3a7f453d-d27d-4725-b371-9f77bb45d9dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abae69a93a014143ab0cebfd1a7be830'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:02.490043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3a7f453d-d27d-4725-b371-9f77bb45d9dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abae69a93a014143ab0cebfd1a7be830'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:02.491566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3a7f453d-d27d-4725-b371-9f77bb45d9dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abae69a93a014143ab0cebfd1a7be830'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:02.499366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-29953f63-f25b-4197-87e1-652ac75a940d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96cdbb1be17c4352ba93c31238c8b96b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:02.508685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-29953f63-f25b-4197-87e1-652ac75a940d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96cdbb1be17c4352ba93c31238c8b96b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:02.509700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-29953f63-f25b-4197-87e1-652ac75a940d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96cdbb1be17c4352ba93c31238c8b96b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:02.513918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1130/2219] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:32:02 2022] DELETE /identity/v3/users/abae69a93a014143ab0cebfd1a7be830 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:32:02.518549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56999f-475c-49eb-84f7-46343522533b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:02.519315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56999f-475c-49eb-84f7-46343522533b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:02.519855 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56999f-475c-49eb-84f7-46343522533b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:02.524038 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-29953f63-f25b-4197-87e1-652ac75a940d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96cdbb1be17c4352ba93c31238c8b96b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:02.526999 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-29953f63-f25b-4197-87e1-652ac75a940d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96cdbb1be17c4352ba93c31238c8b96b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:02.553913 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9c56999f-475c-49eb-84f7-46343522533b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:02.555102 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9c56999f-475c-49eb-84f7-46343522533b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:02.556615 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1090/2220] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:32:02 2022] DELETE /identity/v3/users/96cdbb1be17c4352ba93c31238c8b96b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:32:02.561299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf51860-5822-4bf2-9fb8-d76742393a01 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:02.561731 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf51860-5822-4bf2-9fb8-d76742393a01 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:02.562005 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf51860-5822-4bf2-9fb8-d76742393a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:02.580601 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ebf51860-5822-4bf2-9fb8-d76742393a01 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:02.581939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ebf51860-5822-4bf2-9fb8-d76742393a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:02.628288 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1131/2221] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:02.641026 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56999f-475c-49eb-84f7-46343522533b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:02.645130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9c56999f-475c-49eb-84f7-46343522533b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:02.656055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1091/2222] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:02.667740 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf51860-5822-4bf2-9fb8-d76742393a01 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:02.670422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ebf51860-5822-4bf2-9fb8-d76742393a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:02.773191 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf51860-5822-4bf2-9fb8-d76742393a01 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:02.775148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56999f-475c-49eb-84f7-46343522533b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:02.778527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac87cc-5ee3-4f33-b81f-85886769f0e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:02.779405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac87cc-5ee3-4f33-b81f-85886769f0e7 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:02.779840 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac87cc-5ee3-4f33-b81f-85886769f0e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:02.780291 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac87cc-5ee3-4f33-b81f-85886769f0e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:02.780419 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368a9ee-96a7-4b2e-b809-85ed254b150e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:02.780991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368a9ee-96a7-4b2e-b809-85ed254b150e service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:02.781278 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368a9ee-96a7-4b2e-b809-85ed254b150e service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:02.781640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368a9ee-96a7-4b2e-b809-85ed254b150e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:02.792748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368a9ee-96a7-4b2e-b809-85ed254b150e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:02.794168 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368a9ee-96a7-4b2e-b809-85ed254b150e service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:02.795792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac87cc-5ee3-4f33-b81f-85886769f0e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:02.797414 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac87cc-5ee3-4f33-b81f-85886769f0e7 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:02.804553 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1132/2223] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:32:02.812094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1092/2224] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:32:03.273930 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368a9ee-96a7-4b2e-b809-85ed254b150e service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:03.276059 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c368a9ee-96a7-4b2e-b809-85ed254b150e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:03.290859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368a9ee-96a7-4b2e-b809-85ed254b150e service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:03.297757 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed35dc-89e6-4b66-9232-4fe7f08ac081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:03.298978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed35dc-89e6-4b66-9232-4fe7f08ac081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.299439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed35dc-89e6-4b66-9232-4fe7f08ac081 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.300011 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed35dc-89e6-4b66-9232-4fe7f08ac081 admin admin] PATH_INFO: `/v3/projects/a2b375defd1e41149b8949a808b49dd0` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.301384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed35dc-89e6-4b66-9232-4fe7f08ac081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b375defd1e41149b8949a808b49dd0)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:03.303827 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed35dc-89e6-4b66-9232-4fe7f08ac081 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:03.326963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac87cc-5ee3-4f33-b81f-85886769f0e7 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:03.328896 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-10ac87cc-5ee3-4f33-b81f-85886769f0e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:03.349854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac87cc-5ee3-4f33-b81f-85886769f0e7 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:03.356866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496b031-6004-42cc-b650-b931c330c989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:03.358016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496b031-6004-42cc-b650-b931c330c989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.358466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496b031-6004-42cc-b650-b931c330c989 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.358884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496b031-6004-42cc-b650-b931c330c989 admin admin] PATH_INFO: `/v3/projects/bc8ecae09da549248c002d344f413ed2` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.360412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496b031-6004-42cc-b650-b931c330c989 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc8ecae09da549248c002d344f413ed2)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:03.363236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496b031-6004-42cc-b650-b931c330c989 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:03.406633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-7aed35dc-89e6-4b66-9232-4fe7f08ac081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b375defd1e41149b8949a808b49dd0', 'request_id': 'req-7aed35dc-89e6-4b66-9232-4fe7f08ac081'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:03.412829 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-7aed35dc-89e6-4b66-9232-4fe7f08ac081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b375defd1e41149b8949a808b49dd0', 'request_id': 'req-7aed35dc-89e6-4b66-9232-4fe7f08ac081'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:03.423583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-7aed35dc-89e6-4b66-9232-4fe7f08ac081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b375defd1e41149b8949a808b49dd0', 'request_id': 'req-7aed35dc-89e6-4b66-9232-4fe7f08ac081'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:03.451162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-7aed35dc-89e6-4b66-9232-4fe7f08ac081 admin admin] The token cache is being invalidate because project a2b375defd1e41149b8949a808b49dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:32:03.451485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-7aed35dc-89e6-4b66-9232-4fe7f08ac081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:03.452149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-7aed35dc-89e6-4b66-9232-4fe7f08ac081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:03.455528 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e496b031-6004-42cc-b650-b931c330c989 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc8ecae09da549248c002d344f413ed2', 'request_id': 'req-e496b031-6004-42cc-b650-b931c330c989'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:03.457448 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1133/2225] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:32:03 2022] DELETE /identity/v3/projects/a2b375defd1e41149b8949a808b49dd0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:32:03.458831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee979be9-dbc2-4e4f-ae89-93dba94b213d None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.459121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee979be9-dbc2-4e4f-ae89-93dba94b213d None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.459422 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee979be9-dbc2-4e4f-ae89-93dba94b213d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.462663 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e496b031-6004-42cc-b650-b931c330c989 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc8ecae09da549248c002d344f413ed2', 'request_id': 'req-e496b031-6004-42cc-b650-b931c330c989'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:03.473935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e496b031-6004-42cc-b650-b931c330c989 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc8ecae09da549248c002d344f413ed2', 'request_id': 'req-e496b031-6004-42cc-b650-b931c330c989'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:03.497371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e496b031-6004-42cc-b650-b931c330c989 admin admin] The token cache is being invalidate because project bc8ecae09da549248c002d344f413ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:32:03.497594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e496b031-6004-42cc-b650-b931c330c989 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:03.498622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e496b031-6004-42cc-b650-b931c330c989 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:03.500931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1093/2226] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:32:03 2022] DELETE /identity/v3/projects/bc8ecae09da549248c002d344f413ed2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:32:03.506123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a727443-7068-4f2a-afd9-f0a1749ad612 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.506123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a727443-7068-4f2a-afd9-f0a1749ad612 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.506411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a727443-7068-4f2a-afd9-f0a1749ad612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.525414 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-ee979be9-dbc2-4e4f-ae89-93dba94b213d None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:03.529171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ee979be9-dbc2-4e4f-ae89-93dba94b213d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:03.540782 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-0a727443-7068-4f2a-afd9-f0a1749ad612 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:03.542528 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-0a727443-7068-4f2a-afd9-f0a1749ad612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:03.609451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1134/2227] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:03.617893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1e920-d0b7-4cdf-b3ab-78cb22673232 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.618428 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1e920-d0b7-4cdf-b3ab-78cb22673232 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.618858 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1e920-d0b7-4cdf-b3ab-78cb22673232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.639688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1094/2228] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:03.647323 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-88f1e920-d0b7-4cdf-b3ab-78cb22673232 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:03.648914 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823125e-55a6-4740-a0e3-999af9182908 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.649201 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823125e-55a6-4740-a0e3-999af9182908 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.649475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823125e-55a6-4740-a0e3-999af9182908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.651950 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-88f1e920-d0b7-4cdf-b3ab-78cb22673232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:03.663657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1135/2229] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:32:03.665225 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1e920-d0b7-4cdf-b3ab-78cb22673232 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:03.671506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f823125e-55a6-4740-a0e3-999af9182908 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:03.672693 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f823125e-55a6-4740-a0e3-999af9182908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:03.675703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1e920-d0b7-4cdf-b3ab-78cb22673232 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:03.680892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e475d-53be-43b7-9838-588743220078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:03.681633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e475d-53be-43b7-9838-588743220078 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.681978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e475d-53be-43b7-9838-588743220078 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.682418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e475d-53be-43b7-9838-588743220078 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.683307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e475d-53be-43b7-9838-588743220078 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:32:03.683683 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e475d-53be-43b7-9838-588743220078 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:03.687290 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1095/2230] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:03.687406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e475d-53be-43b7-9838-588743220078 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:03.688877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823125e-55a6-4740-a0e3-999af9182908 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:03.698610 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1136/2231] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:32:03.698971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823125e-55a6-4740-a0e3-999af9182908 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:03.701934 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e475d-53be-43b7-9838-588743220078 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:03.704451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a939535-1841-46df-af65-d828388d5e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:03.705142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a939535-1841-46df-af65-d828388d5e41 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.705487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a939535-1841-46df-af65-d828388d5e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.706781 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a939535-1841-46df-af65-d828388d5e41 admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.708837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a939535-1841-46df-af65-d828388d5e41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:32:03.709777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a939535-1841-46df-af65-d828388d5e41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:03.711915 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e475d-53be-43b7-9838-588743220078 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:03.712041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a939535-1841-46df-af65-d828388d5e41 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:03.717126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc923a43-56fc-4deb-8d63-9fc1149c1ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:03.717774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc923a43-56fc-4deb-8d63-9fc1149c1ddc admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.718056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc923a43-56fc-4deb-8d63-9fc1149c1ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.718331 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc923a43-56fc-4deb-8d63-9fc1149c1ddc admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.719012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc923a43-56fc-4deb-8d63-9fc1149c1ddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:03.721105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc923a43-56fc-4deb-8d63-9fc1149c1ddc admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:03.727968 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1096/2232] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:32:03.728946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a939535-1841-46df-af65-d828388d5e41 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:03.739342 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a939535-1841-46df-af65-d828388d5e41 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:03.746601 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b1b00-d92b-47df-addf-0e49ffac7e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:03.747700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b1b00-d92b-47df-addf-0e49ffac7e8f admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.748233 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b1b00-d92b-47df-addf-0e49ffac7e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.748665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b1b00-d92b-47df-addf-0e49ffac7e8f admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.749502 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b1b00-d92b-47df-addf-0e49ffac7e8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:03.749742 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1137/2233] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:32:03 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:32:03.752410 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b1b00-d92b-47df-addf-0e49ffac7e8f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:03.753980 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc923a43-56fc-4deb-8d63-9fc1149c1ddc admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:03.788779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1097/2234] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:32:03 2022] POST /identity/v3/projects => generated 391 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:32:03.790195 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b1b00-d92b-47df-addf-0e49ffac7e8f admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:03.818291 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc923a43-56fc-4deb-8d63-9fc1149c1ddc admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:03.862019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b1b00-d92b-47df-addf-0e49ffac7e8f admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:03.867016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961fe4c-ac84-424f-a1e9-9004ddc14633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:03.867728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961fe4c-ac84-424f-a1e9-9004ddc14633 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.868016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961fe4c-ac84-424f-a1e9-9004ddc14633 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.868299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961fe4c-ac84-424f-a1e9-9004ddc14633 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.868985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961fe4c-ac84-424f-a1e9-9004ddc14633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:03.870944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961fe4c-ac84-424f-a1e9-9004ddc14633 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:03.877160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecf587-79cd-4480-90cc-ee0303f009e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:03.877849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecf587-79cd-4480-90cc-ee0303f009e5 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.878134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecf587-79cd-4480-90cc-ee0303f009e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.878413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecf587-79cd-4480-90cc-ee0303f009e5 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.879102 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecf587-79cd-4480-90cc-ee0303f009e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:03.881030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecf587-79cd-4480-90cc-ee0303f009e5 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:03.891945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1098/2235] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:32:03 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:32:03.896181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961fe4c-ac84-424f-a1e9-9004ddc14633 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:03.907512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961fe4c-ac84-424f-a1e9-9004ddc14633 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:03.911651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1138/2236] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:32:03 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:32:03.912704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecf587-79cd-4480-90cc-ee0303f009e5 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:03.913052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807b511-67b0-4e44-b469-c805402d9301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:03.913818 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807b511-67b0-4e44-b469-c805402d9301 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.914171 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807b511-67b0-4e44-b469-c805402d9301 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.914513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807b511-67b0-4e44-b469-c805402d9301 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.915826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807b511-67b0-4e44-b469-c805402d9301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:03.917600 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807b511-67b0-4e44-b469-c805402d9301 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:03.923487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecf587-79cd-4480-90cc-ee0303f009e5 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:03.928214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a2b84-851b-4bac-976c-eb88b7568512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:03.931985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1099/2237] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:32:03.932137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a2b84-851b-4bac-976c-eb88b7568512 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.932137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a2b84-851b-4bac-976c-eb88b7568512 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.932137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a2b84-851b-4bac-976c-eb88b7568512 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.932137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a2b84-851b-4bac-976c-eb88b7568512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:03.932137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a2b84-851b-4bac-976c-eb88b7568512 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:03.932802 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807b511-67b0-4e44-b469-c805402d9301 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:03.944996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1139/2238] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:32:03.945241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807b511-67b0-4e44-b469-c805402d9301 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:03.946300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a2b84-851b-4bac-976c-eb88b7568512 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:03.949419 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c594b1-332f-459a-9461-99eb968e994c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:03.950160 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c594b1-332f-459a-9461-99eb968e994c admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.950512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c594b1-332f-459a-9461-99eb968e994c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.950863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c594b1-332f-459a-9461-99eb968e994c admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.951966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c594b1-332f-459a-9461-99eb968e994c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:03.953736 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c594b1-332f-459a-9461-99eb968e994c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:03.956950 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a2b84-851b-4bac-976c-eb88b7568512 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:03.971244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1100/2239] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:32:03.973410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e7d6d-3d98-40cf-a419-56f694e38e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:03.974389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c594b1-332f-459a-9461-99eb968e994c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:03.974996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e7d6d-3d98-40cf-a419-56f694e38e48 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.975867 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e7d6d-3d98-40cf-a419-56f694e38e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.976335 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e7d6d-3d98-40cf-a419-56f694e38e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.977516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e7d6d-3d98-40cf-a419-56f694e38e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:03.980460 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e7d6d-3d98-40cf-a419-56f694e38e48 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:03.987055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c594b1-332f-459a-9461-99eb968e994c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:03.992039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16e6ea-33c2-4d68-8516-1c99c2018882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:03.996355 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1140/2240] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:32:03.996500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16e6ea-33c2-4d68-8516-1c99c2018882 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:03.996500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16e6ea-33c2-4d68-8516-1c99c2018882 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:03.996500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16e6ea-33c2-4d68-8516-1c99c2018882 admin admin] PATH_INFO: `/v3/projects/142400911d964c1bb61da3976f5e0f0f/users/9d9e470e2a1845f49223b5f0768c3db3/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:03.996758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e7d6d-3d98-40cf-a419-56f694e38e48 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:04.010039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16e6ea-33c2-4d68-8516-1c99c2018882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=142400911d964c1bb61da3976f5e0f0f, user_id=9d9e470e2a1845f49223b5f0768c3db3, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:04.013343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16e6ea-33c2-4d68-8516-1c99c2018882 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:04.029752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e7d6d-3d98-40cf-a419-56f694e38e48 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:04.043879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1101/2241] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:32:03 2022] PUT /identity/v3/projects/142400911d964c1bb61da3976f5e0f0f/users/9d9e470e2a1845f49223b5f0768c3db3/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:32:04.047621 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9440eea-f4e5-41cf-bc0e-14eabc199d29 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:04.047954 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9440eea-f4e5-41cf-bc0e-14eabc199d29 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:04.048479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9440eea-f4e5-41cf-bc0e-14eabc199d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:04.109439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e527b-cc54-4e85-873e-a616a3cea4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:04.110196 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e527b-cc54-4e85-873e-a616a3cea4fc admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:04.110477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e527b-cc54-4e85-873e-a616a3cea4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:04.110752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e527b-cc54-4e85-873e-a616a3cea4fc admin admin] PATH_INFO: `/v3/projects/70c9f13a55b04467a9c84c624ae9bf5d/users/933e9db5cbe14f6596f38fc5a247e243/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:04.117270 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f9440eea-f4e5-41cf-bc0e-14eabc199d29 None None] MFA Rules not processed for user `9d9e470e2a1845f49223b5f0768c3db3`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:04.119080 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f9440eea-f4e5-41cf-bc0e-14eabc199d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:04.141278 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e527b-cc54-4e85-873e-a616a3cea4fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c9f13a55b04467a9c84c624ae9bf5d, user_id=933e9db5cbe14f6596f38fc5a247e243, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:04.151644 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e527b-cc54-4e85-873e-a616a3cea4fc admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:04.170488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1141/2242] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:32:03 2022] PUT /identity/v3/projects/70c9f13a55b04467a9c84c624ae9bf5d/users/933e9db5cbe14f6596f38fc5a247e243/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:32:04.180516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ccf7c-1bee-4fc3-a252-792ccf6c8a10 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:04.180697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ccf7c-1bee-4fc3-a252-792ccf6c8a10 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:04.180850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ccf7c-1bee-4fc3-a252-792ccf6c8a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:04.208148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-e19ccf7c-1bee-4fc3-a252-792ccf6c8a10 None None] MFA Rules not processed for user `933e9db5cbe14f6596f38fc5a247e243`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:04.209607 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e19ccf7c-1bee-4fc3-a252-792ccf6c8a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:04.212368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1102/2243] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:04.224693 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9440eea-f4e5-41cf-bc0e-14eabc199d29 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:04.226515 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f9440eea-f4e5-41cf-bc0e-14eabc199d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:04.297999 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1142/2244] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:32:04.307162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ccf7c-1bee-4fc3-a252-792ccf6c8a10 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:04.308976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e19ccf7c-1bee-4fc3-a252-792ccf6c8a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:04.309342 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9440eea-f4e5-41cf-bc0e-14eabc199d29 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:04.380217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ccf7c-1bee-4fc3-a252-792ccf6c8a10 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:04.380920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a6485-24ae-4245-af43-41ada936b0a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:04.381707 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a6485-24ae-4245-af43-41ada936b0a0 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:04.382329 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a6485-24ae-4245-af43-41ada936b0a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:04.382680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a6485-24ae-4245-af43-41ada936b0a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:04.384695 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f382fdd-8f32-4b4b-b058-97c0052c0654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:04.385958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f382fdd-8f32-4b4b-b058-97c0052c0654 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:04.386321 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f382fdd-8f32-4b4b-b058-97c0052c0654 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:04.386674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f382fdd-8f32-4b4b-b058-97c0052c0654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:04.398823 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f382fdd-8f32-4b4b-b058-97c0052c0654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:04.400762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f382fdd-8f32-4b4b-b058-97c0052c0654 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:04.426942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1143/2245] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:32:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:32:04.441451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f382fdd-8f32-4b4b-b058-97c0052c0654 service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:04.476497 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a6485-24ae-4245-af43-41ada936b0a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:04.478516 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a6485-24ae-4245-af43-41ada936b0a0 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:04.490724 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1103/2246] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:32:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:32:04.575758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f382fdd-8f32-4b4b-b058-97c0052c0654 service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:04.585979 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbffc5-7d5f-4c9c-9efb-aeac3798b0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:04.587111 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbffc5-7d5f-4c9c-9efb-aeac3798b0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:04.587581 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbffc5-7d5f-4c9c-9efb-aeac3798b0de admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:04.589132 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbffc5-7d5f-4c9c-9efb-aeac3798b0de admin admin] PATH_INFO: `/v3/users/933e9db5cbe14f6596f38fc5a247e243` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:04.591410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdbffc5-7d5f-4c9c-9efb-aeac3798b0de admin admin] RBAC: Authorizing `identity:delete_user(user_id=933e9db5cbe14f6596f38fc5a247e243)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:04.594107 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdbffc5-7d5f-4c9c-9efb-aeac3798b0de admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:04.669894 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8fdbffc5-7d5f-4c9c-9efb-aeac3798b0de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '933e9db5cbe14f6596f38fc5a247e243'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:04.671169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8fdbffc5-7d5f-4c9c-9efb-aeac3798b0de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '933e9db5cbe14f6596f38fc5a247e243'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:04.685311 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8fdbffc5-7d5f-4c9c-9efb-aeac3798b0de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '933e9db5cbe14f6596f38fc5a247e243'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:04.695513 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8fdbffc5-7d5f-4c9c-9efb-aeac3798b0de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '933e9db5cbe14f6596f38fc5a247e243'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:04.697403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8fdbffc5-7d5f-4c9c-9efb-aeac3798b0de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '933e9db5cbe14f6596f38fc5a247e243'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:04.752771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1144/2247] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:32:04 2022] DELETE /identity/v3/users/933e9db5cbe14f6596f38fc5a247e243 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:32:04.754752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06009d92-1e4b-4e98-abda-9bd75d84df4b None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:04.755350 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06009d92-1e4b-4e98-abda-9bd75d84df4b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:04.755779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06009d92-1e4b-4e98-abda-9bd75d84df4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:04.785987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-06009d92-1e4b-4e98-abda-9bd75d84df4b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:04.787756 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-06009d92-1e4b-4e98-abda-9bd75d84df4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:04.889741 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1104/2248] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:04.895837 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbffc5-7d5f-4c9c-9efb-aeac3798b0de admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:04.898018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-8fdbffc5-7d5f-4c9c-9efb-aeac3798b0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:04.973844 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbffc5-7d5f-4c9c-9efb-aeac3798b0de admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:04.977556 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44a903-d985-4e57-a828-66da08877030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:04.978199 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44a903-d985-4e57-a828-66da08877030 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:04.978491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44a903-d985-4e57-a828-66da08877030 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:04.978757 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44a903-d985-4e57-a828-66da08877030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:04.989922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44a903-d985-4e57-a828-66da08877030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:04.991390 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44a903-d985-4e57-a828-66da08877030 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:05.002055 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1145/2249] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:32:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:32:05.269075 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06009d92-1e4b-4e98-abda-9bd75d84df4b None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:05.271773 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-06009d92-1e4b-4e98-abda-9bd75d84df4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:05.284347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06009d92-1e4b-4e98-abda-9bd75d84df4b None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:05.291926 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84932a-5bc0-4512-926b-ae57db3ad4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:05.293308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84932a-5bc0-4512-926b-ae57db3ad4e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:05.293586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84932a-5bc0-4512-926b-ae57db3ad4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:05.293858 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84932a-5bc0-4512-926b-ae57db3ad4e7 admin admin] PATH_INFO: `/v3/projects/70c9f13a55b04467a9c84c624ae9bf5d` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:05.295077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84932a-5bc0-4512-926b-ae57db3ad4e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70c9f13a55b04467a9c84c624ae9bf5d)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:05.297001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84932a-5bc0-4512-926b-ae57db3ad4e7 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:05.344773 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-eb84932a-5bc0-4512-926b-ae57db3ad4e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70c9f13a55b04467a9c84c624ae9bf5d', 'request_id': 'req-eb84932a-5bc0-4512-926b-ae57db3ad4e7'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:05.352103 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-eb84932a-5bc0-4512-926b-ae57db3ad4e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70c9f13a55b04467a9c84c624ae9bf5d', 'request_id': 'req-eb84932a-5bc0-4512-926b-ae57db3ad4e7'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:05.360782 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-eb84932a-5bc0-4512-926b-ae57db3ad4e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70c9f13a55b04467a9c84c624ae9bf5d', 'request_id': 'req-eb84932a-5bc0-4512-926b-ae57db3ad4e7'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:05.373416 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-eb84932a-5bc0-4512-926b-ae57db3ad4e7 admin admin] The token cache is being invalidate because project 70c9f13a55b04467a9c84c624ae9bf5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:32:05.373972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-eb84932a-5bc0-4512-926b-ae57db3ad4e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:05.374813 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-eb84932a-5bc0-4512-926b-ae57db3ad4e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:05.380179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1105/2250] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:32:05 2022] DELETE /identity/v3/projects/70c9f13a55b04467a9c84c624ae9bf5d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:32:05.382312 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117f72e-80b1-45fe-ad32-04ef67e905c4 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:05.382776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117f72e-80b1-45fe-ad32-04ef67e905c4 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:05.382869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117f72e-80b1-45fe-ad32-04ef67e905c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:05.401982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-d117f72e-80b1-45fe-ad32-04ef67e905c4 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:05.403231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-d117f72e-80b1-45fe-ad32-04ef67e905c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:05.483315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1146/2251] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:05.490478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03384ce-9246-449c-8d4a-184070d92fb6 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:05.490972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03384ce-9246-449c-8d4a-184070d92fb6 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:05.491065 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03384ce-9246-449c-8d4a-184070d92fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:05.517866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f03384ce-9246-449c-8d4a-184070d92fb6 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:05.519262 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f03384ce-9246-449c-8d4a-184070d92fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:05.528257 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1106/2252] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:05.532821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117f72e-80b1-45fe-ad32-04ef67e905c4 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:05.542537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117f72e-80b1-45fe-ad32-04ef67e905c4 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:05.547634 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0534c1a-2e35-4431-ae4f-8db89bb053b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:05.548299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534c1a-2e35-4431-ae4f-8db89bb053b4 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:05.548583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534c1a-2e35-4431-ae4f-8db89bb053b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:05.548862 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534c1a-2e35-4431-ae4f-8db89bb053b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:05.549635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0534c1a-2e35-4431-ae4f-8db89bb053b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:32:05.549941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0534c1a-2e35-4431-ae4f-8db89bb053b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:05.551660 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0534c1a-2e35-4431-ae4f-8db89bb053b4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:05.564856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1147/2253] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:32:05.575075 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03384ce-9246-449c-8d4a-184070d92fb6 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:05.592351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03384ce-9246-449c-8d4a-184070d92fb6 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:05.599861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bd13f-7173-409e-a8d8-02a6f16af0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:05.600257 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bd13f-7173-409e-a8d8-02a6f16af0ce admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:05.600663 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bd13f-7173-409e-a8d8-02a6f16af0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:05.601040 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bd13f-7173-409e-a8d8-02a6f16af0ce admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:05.602420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bd13f-7173-409e-a8d8-02a6f16af0ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:05.604698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bd13f-7173-409e-a8d8-02a6f16af0ce admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:05.634011 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1107/2254] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:32:05 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:32:05.638459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0534c1a-2e35-4431-ae4f-8db89bb053b4 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:05.649221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0534c1a-2e35-4431-ae4f-8db89bb053b4 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:05.715547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a534306-398c-419b-86f8-7786a0d4dc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:05.716232 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a534306-398c-419b-86f8-7786a0d4dc13 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:05.716517 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a534306-398c-419b-86f8-7786a0d4dc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:05.716794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a534306-398c-419b-86f8-7786a0d4dc13 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:05.717591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a534306-398c-419b-86f8-7786a0d4dc13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:05.719759 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a534306-398c-419b-86f8-7786a0d4dc13 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:05.739848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1148/2255] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:32:05 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:32:05.743984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bd13f-7173-409e-a8d8-02a6f16af0ce admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:05.755663 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bd13f-7173-409e-a8d8-02a6f16af0ce admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:05.761013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f48e09-772e-4e30-b1fd-576b5bd2c6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:05.761677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f48e09-772e-4e30-b1fd-576b5bd2c6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:05.761955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f48e09-772e-4e30-b1fd-576b5bd2c6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:05.762225 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f48e09-772e-4e30-b1fd-576b5bd2c6e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:05.762944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f48e09-772e-4e30-b1fd-576b5bd2c6e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:05.764598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f48e09-772e-4e30-b1fd-576b5bd2c6e8 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:05.778493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1108/2256] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:32:05.779857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a534306-398c-419b-86f8-7786a0d4dc13 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:05.790273 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a534306-398c-419b-86f8-7786a0d4dc13 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:05.794930 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f013ec4-6353-488c-815d-5222e50774cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:05.795629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f013ec4-6353-488c-815d-5222e50774cb admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:05.795905 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f013ec4-6353-488c-815d-5222e50774cb admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:05.796180 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f013ec4-6353-488c-815d-5222e50774cb admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:05.796893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f013ec4-6353-488c-815d-5222e50774cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:05.798445 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f013ec4-6353-488c-815d-5222e50774cb admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:05.808370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1149/2257] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:32:05.812569 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f48e09-772e-4e30-b1fd-576b5bd2c6e8 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:05.822405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f48e09-772e-4e30-b1fd-576b5bd2c6e8 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:05.827656 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5ed4b-3bb9-4b61-b666-0b892b6512b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:05.828525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5ed4b-3bb9-4b61-b666-0b892b6512b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:05.828807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5ed4b-3bb9-4b61-b666-0b892b6512b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:05.829081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5ed4b-3bb9-4b61-b666-0b892b6512b0 admin admin] PATH_INFO: `/v3/projects/366da93a37f54180a7c2226d90a2356d/users/4dfc640ee055498e9af6151c5d3320d8/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:05.851613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5ed4b-3bb9-4b61-b666-0b892b6512b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=366da93a37f54180a7c2226d90a2356d, user_id=4dfc640ee055498e9af6151c5d3320d8, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:05.854205 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5ed4b-3bb9-4b61-b666-0b892b6512b0 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:05.867791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1109/2258] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:32:05 2022] PUT /identity/v3/projects/366da93a37f54180a7c2226d90a2356d/users/4dfc640ee055498e9af6151c5d3320d8/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:32:05.878928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c436b-82c6-43e1-83c3-13668d98ee6f None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:05.879345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c436b-82c6-43e1-83c3-13668d98ee6f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:05.879708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c436b-82c6-43e1-83c3-13668d98ee6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:05.907447 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2a9c436b-82c6-43e1-83c3-13668d98ee6f None None] MFA Rules not processed for user `4dfc640ee055498e9af6151c5d3320d8`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:05.908772 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2a9c436b-82c6-43e1-83c3-13668d98ee6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:05.984698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1150/2259] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:05.995800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5ed4b-3bb9-4b61-b666-0b892b6512b0 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:05.997572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c4a5ed4b-3bb9-4b61-b666-0b892b6512b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:06.054973 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5ed4b-3bb9-4b61-b666-0b892b6512b0 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:06.059620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249797a1-7e44-4dfc-bc0a-0d986367a3ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:06.060371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249797a1-7e44-4dfc-bc0a-0d986367a3ac service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:06.060664 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249797a1-7e44-4dfc-bc0a-0d986367a3ac service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:06.060971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249797a1-7e44-4dfc-bc0a-0d986367a3ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:06.069486 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249797a1-7e44-4dfc-bc0a-0d986367a3ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:06.071000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249797a1-7e44-4dfc-bc0a-0d986367a3ac service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:06.080455 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1110/2260] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:32:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:32:07.124915 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c436b-82c6-43e1-83c3-13668d98ee6f None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:07.126550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2a9c436b-82c6-43e1-83c3-13668d98ee6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:07.181417 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c436b-82c6-43e1-83c3-13668d98ee6f None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:07.184980 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0b6f4-466e-40f0-9950-497e72d7b544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:07.185628 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0b6f4-466e-40f0-9950-497e72d7b544 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:07.185911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0b6f4-466e-40f0-9950-497e72d7b544 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:07.186189 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0b6f4-466e-40f0-9950-497e72d7b544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:07.187941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-8bb0b6f4-466e-40f0-9950-497e72d7b544 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:07.246175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0b6f4-466e-40f0-9950-497e72d7b544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:07.247618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0b6f4-466e-40f0-9950-497e72d7b544 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:07.257159 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1151/2261] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:32:07 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:32:10.559709 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a4e75-dadd-4cd4-a9a8-e535cbb6d5a6 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:10.560251 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a4e75-dadd-4cd4-a9a8-e535cbb6d5a6 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:10.560251 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a4e75-dadd-4cd4-a9a8-e535cbb6d5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:10.580756 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-8d7a4e75-dadd-4cd4-a9a8-e535cbb6d5a6 None None] MFA Rules not processed for user `4dfc640ee055498e9af6151c5d3320d8`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:10.581695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-8d7a4e75-dadd-4cd4-a9a8-e535cbb6d5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:10.593170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1111/2262] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:20.180041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d308889-346d-49de-ac1d-7be4dd979336 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:20.180618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d308889-346d-49de-ac1d-7be4dd979336 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:20.180618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d308889-346d-49de-ac1d-7be4dd979336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:20.198837 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9d308889-346d-49de-ac1d-7be4dd979336 None None] MFA Rules not processed for user `acae3ad19a7e45b9877520dff8f5b995`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:20.200066 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9d308889-346d-49de-ac1d-7be4dd979336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:20.261900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1152/2263] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:24.072004 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a4e75-dadd-4cd4-a9a8-e535cbb6d5a6 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:24.074176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-8d7a4e75-dadd-4cd4-a9a8-e535cbb6d5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:24.171971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a4e75-dadd-4cd4-a9a8-e535cbb6d5a6 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:24.176784 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a12907-4e0f-4c68-a0d6-a0dd221f3a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:24.177717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a12907-4e0f-4c68-a0d6-a0dd221f3a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:24.178217 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a12907-4e0f-4c68-a0d6-a0dd221f3a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:24.178602 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a12907-4e0f-4c68-a0d6-a0dd221f3a29 admin admin] PATH_INFO: `/v3/users/9d9e470e2a1845f49223b5f0768c3db3` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:24.179888 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a12907-4e0f-4c68-a0d6-a0dd221f3a29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d9e470e2a1845f49223b5f0768c3db3)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:24.181863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a12907-4e0f-4c68-a0d6-a0dd221f3a29 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:24.244838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c8a12907-4e0f-4c68-a0d6-a0dd221f3a29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d9e470e2a1845f49223b5f0768c3db3'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:24.257632 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c8a12907-4e0f-4c68-a0d6-a0dd221f3a29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d9e470e2a1845f49223b5f0768c3db3'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:24.258420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c8a12907-4e0f-4c68-a0d6-a0dd221f3a29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d9e470e2a1845f49223b5f0768c3db3'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:24.279586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c8a12907-4e0f-4c68-a0d6-a0dd221f3a29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d9e470e2a1845f49223b5f0768c3db3'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:24.280443 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c8a12907-4e0f-4c68-a0d6-a0dd221f3a29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d9e470e2a1845f49223b5f0768c3db3'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:24.307837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1112/2264] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:32:24 2022] DELETE /identity/v3/users/9d9e470e2a1845f49223b5f0768c3db3 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:32:24.312809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771f99d-82f2-4880-817a-3089a5bf1cef None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:24.313327 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771f99d-82f2-4880-817a-3089a5bf1cef None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:24.313425 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771f99d-82f2-4880-817a-3089a5bf1cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:24.334811 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9771f99d-82f2-4880-817a-3089a5bf1cef None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:24.336354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9771f99d-82f2-4880-817a-3089a5bf1cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:24.409158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1153/2265] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:24.418919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a12907-4e0f-4c68-a0d6-a0dd221f3a29 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:24.420400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c8a12907-4e0f-4c68-a0d6-a0dd221f3a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:24.487694 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a12907-4e0f-4c68-a0d6-a0dd221f3a29 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:24.491514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbc2d7-e9b8-4198-8944-c3a5088d5a2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:24.492205 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbc2d7-e9b8-4198-8944-c3a5088d5a2e service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:24.492494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbc2d7-e9b8-4198-8944-c3a5088d5a2e service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:24.492779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbc2d7-e9b8-4198-8944-c3a5088d5a2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:24.503431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbc2d7-e9b8-4198-8944-c3a5088d5a2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:24.504877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbc2d7-e9b8-4198-8944-c3a5088d5a2e service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:24.515874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1113/2266] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:32:24.878493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771f99d-82f2-4880-817a-3089a5bf1cef None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:24.879965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9771f99d-82f2-4880-817a-3089a5bf1cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:24.896618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771f99d-82f2-4880-817a-3089a5bf1cef None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:24.903474 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:24.904461 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:24.904843 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:24.905340 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] PATH_INFO: `/v3/projects/142400911d964c1bb61da3976f5e0f0f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:24.906521 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=142400911d964c1bb61da3976f5e0f0f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:24.908669 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:24.956469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '142400911d964c1bb61da3976f5e0f0f', 'request_id': 'req-1864dedd-5221-41f7-8170-186759df1431'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:24.961655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '142400911d964c1bb61da3976f5e0f0f', 'request_id': 'req-1864dedd-5221-41f7-8170-186759df1431'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:24.968760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '142400911d964c1bb61da3976f5e0f0f', 'request_id': 'req-1864dedd-5221-41f7-8170-186759df1431'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:24.979848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] The token cache is being invalidate because project 142400911d964c1bb61da3976f5e0f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:32:24.980130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:24.980753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:24.982261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1154/2267] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:32:24 2022] DELETE /identity/v3/projects/142400911d964c1bb61da3976f5e0f0f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:32:35.792814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86a0c3-3e77-4dae-9916-18ab8c08cfb6 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:35.793339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86a0c3-3e77-4dae-9916-18ab8c08cfb6 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:35.793424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86a0c3-3e77-4dae-9916-18ab8c08cfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:35.889600 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-2a86a0c3-3e77-4dae-9916-18ab8c08cfb6 None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:35.892145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-2a86a0c3-3e77-4dae-9916-18ab8c08cfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:35.993308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1114/2268] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:32:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:32:36.046294 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:36.047806 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:36.142764 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864dedd-5221-41f7-8170-186759df1431 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:36.147901 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3509c8-c086-4005-ab33-e8e8fb8da5bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:36.148565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3509c8-c086-4005-ab33-e8e8fb8da5bd service swift] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:36.148845 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3509c8-c086-4005-ab33-e8e8fb8da5bd service swift] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:36.149117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3509c8-c086-4005-ab33-e8e8fb8da5bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:36.157539 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3509c8-c086-4005-ab33-e8e8fb8da5bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:36.159271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3509c8-c086-4005-ab33-e8e8fb8da5bd service swift] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:36.167572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1155/2269] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:32:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 18:32:36.707640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c5a6f-fb68-4349-94c8-2e3c72695b46 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:36.707930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c5a6f-fb68-4349-94c8-2e3c72695b46 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:36.708207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c5a6f-fb68-4349-94c8-2e3c72695b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:36.726983 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-211c5a6f-fb68-4349-94c8-2e3c72695b46 None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:36.728932 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-211c5a6f-fb68-4349-94c8-2e3c72695b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:36.736278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1115/2270] 10.222.0.25 () {58 vars in 967 bytes} [Mon May 9 18:32:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:36.749134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3509c8-c086-4005-ab33-e8e8fb8da5bd service swift] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:36.759653 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3509c8-c086-4005-ab33-e8e8fb8da5bd service swift] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:36.764466 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d0f0b-0bcc-4449-8837-362ad4916c2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:36.765089 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d0f0b-0bcc-4449-8837-362ad4916c2d service swift] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:36.765366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d0f0b-0bcc-4449-8837-362ad4916c2d service swift] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:36.765638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d0f0b-0bcc-4449-8837-362ad4916c2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:36.773329 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d0f0b-0bcc-4449-8837-362ad4916c2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:36.774806 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d0f0b-0bcc-4449-8837-362ad4916c2d service swift] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:36.783016 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1156/2271] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:32:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 18:32:36.994526 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cefb68-6874-451d-b575-e34c02fe1bc1 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:36.994964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cefb68-6874-451d-b575-e34c02fe1bc1 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:36.995072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cefb68-6874-451d-b575-e34c02fe1bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:37.013511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-53cefb68-6874-451d-b575-e34c02fe1bc1 None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:37.014640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-53cefb68-6874-451d-b575-e34c02fe1bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:37.021942 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1116/2272] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:32:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:37.034122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d0f0b-0bcc-4449-8837-362ad4916c2d service swift] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:37.044550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d0f0b-0bcc-4449-8837-362ad4916c2d service swift] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:37.049493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa439d5-395a-4122-8a1d-3992ee0361b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:37.050129 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa439d5-395a-4122-8a1d-3992ee0361b5 service swift] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:37.050621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa439d5-395a-4122-8a1d-3992ee0361b5 service swift] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:37.051051 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa439d5-395a-4122-8a1d-3992ee0361b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:37.059401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa439d5-395a-4122-8a1d-3992ee0361b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:37.060954 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa439d5-395a-4122-8a1d-3992ee0361b5 service swift] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:37.068852 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1157/2273] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:32:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 09 18:32:37.393045 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1509e0-d5c0-4ad2-bb1b-317eaea258d0 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:37.393333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1509e0-d5c0-4ad2-bb1b-317eaea258d0 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:37.393612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1509e0-d5c0-4ad2-bb1b-317eaea258d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:37.410751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-3a1509e0-d5c0-4ad2-bb1b-317eaea258d0 None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:37.411988 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-3a1509e0-d5c0-4ad2-bb1b-317eaea258d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:37.418975 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1117/2274] 10.222.0.25 () {58 vars in 967 bytes} [Mon May 9 18:32:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:37.430629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa439d5-395a-4122-8a1d-3992ee0361b5 service swift] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:37.441418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa439d5-395a-4122-8a1d-3992ee0361b5 service swift] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:37.446370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32ba6f-99b9-4331-8dfd-63cbd990a637 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:37.447654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32ba6f-99b9-4331-8dfd-63cbd990a637 service swift] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:37.447940 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32ba6f-99b9-4331-8dfd-63cbd990a637 service swift] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:37.448216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32ba6f-99b9-4331-8dfd-63cbd990a637 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:37.456044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32ba6f-99b9-4331-8dfd-63cbd990a637 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:37.457650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32ba6f-99b9-4331-8dfd-63cbd990a637 service swift] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:37.465688 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1158/2275] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:32:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 18:32:39.551217 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1509e0-d5c0-4ad2-bb1b-317eaea258d0 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:39.552632 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-3a1509e0-d5c0-4ad2-bb1b-317eaea258d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:39.615943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1509e0-d5c0-4ad2-bb1b-317eaea258d0 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:39.621255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fa08b-1900-429f-81a5-dc9bba101a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:39.622127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fa08b-1900-429f-81a5-dc9bba101a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:39.622410 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fa08b-1900-429f-81a5-dc9bba101a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:39.622686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fa08b-1900-429f-81a5-dc9bba101a4b admin admin] PATH_INFO: `/v3/users/4dfc640ee055498e9af6151c5d3320d8` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:39.623895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fa08b-1900-429f-81a5-dc9bba101a4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dfc640ee055498e9af6151c5d3320d8)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:39.626069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fa08b-1900-429f-81a5-dc9bba101a4b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:39.672516 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-303fa08b-1900-429f-81a5-dc9bba101a4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dfc640ee055498e9af6151c5d3320d8'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:39.680282 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-303fa08b-1900-429f-81a5-dc9bba101a4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dfc640ee055498e9af6151c5d3320d8'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:39.680976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-303fa08b-1900-429f-81a5-dc9bba101a4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dfc640ee055498e9af6151c5d3320d8'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:39.688153 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-303fa08b-1900-429f-81a5-dc9bba101a4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dfc640ee055498e9af6151c5d3320d8'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:39.688761 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-303fa08b-1900-429f-81a5-dc9bba101a4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dfc640ee055498e9af6151c5d3320d8'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:39.705692 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1118/2276] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:32:39 2022] DELETE /identity/v3/users/4dfc640ee055498e9af6151c5d3320d8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:32:39.710253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214fe30-26b7-4311-831b-164463572139 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:39.710704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214fe30-26b7-4311-831b-164463572139 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:39.710822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214fe30-26b7-4311-831b-164463572139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:39.728004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-1214fe30-26b7-4311-831b-164463572139 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:39.729782 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-1214fe30-26b7-4311-831b-164463572139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:39.785364 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1159/2277] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:39.795727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fa08b-1900-429f-81a5-dc9bba101a4b admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:39.797098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-303fa08b-1900-429f-81a5-dc9bba101a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:39.851256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fa08b-1900-429f-81a5-dc9bba101a4b admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:39.855426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30ac39-daae-49cc-b71b-a0028115494b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:39.856062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30ac39-daae-49cc-b71b-a0028115494b service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:39.856345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30ac39-daae-49cc-b71b-a0028115494b service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:39.856639 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30ac39-daae-49cc-b71b-a0028115494b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:39.865471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30ac39-daae-49cc-b71b-a0028115494b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:39.866908 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30ac39-daae-49cc-b71b-a0028115494b service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:39.876866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1119/2278] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:32:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:32:40.042433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214fe30-26b7-4311-831b-164463572139 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:40.043931 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-1214fe30-26b7-4311-831b-164463572139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:40.056777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214fe30-26b7-4311-831b-164463572139 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:40.061002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:40.061863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:40.062147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:40.062423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] PATH_INFO: `/v3/projects/366da93a37f54180a7c2226d90a2356d` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:40.063646 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=366da93a37f54180a7c2226d90a2356d)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:40.065514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:40.111751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '366da93a37f54180a7c2226d90a2356d', 'request_id': 'req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:40.117237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '366da93a37f54180a7c2226d90a2356d', 'request_id': 'req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:40.124392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '366da93a37f54180a7c2226d90a2356d', 'request_id': 'req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:40.135792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] The token cache is being invalidate because project 366da93a37f54180a7c2226d90a2356d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:32:40.136076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:40.136696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:32:40.138246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1160/2279] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:32:40 2022] DELETE /identity/v3/projects/366da93a37f54180a7c2226d90a2356d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:32:46.884992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005b935-fe43-47e0-8292-5125e7d24496 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:46.885492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005b935-fe43-47e0-8292-5125e7d24496 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:46.885604 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005b935-fe43-47e0-8292-5125e7d24496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:46.902550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-2005b935-fe43-47e0-8292-5125e7d24496 None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:32:46.903804 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-2005b935-fe43-47e0-8292-5125e7d24496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:46.957310 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1120/2280] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:32:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:32:46.974315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:46.975796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:47.042583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d8d13-7554-488f-8e6a-54c65bfd3cb9 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:47.048043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a079163-93bb-4c51-aa73-58bda0aa3341 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:47.052412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a079163-93bb-4c51-aa73-58bda0aa3341 service swift] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:47.052701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a079163-93bb-4c51-aa73-58bda0aa3341 service swift] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:47.052987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a079163-93bb-4c51-aa73-58bda0aa3341 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:47.061971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a079163-93bb-4c51-aa73-58bda0aa3341 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:47.063587 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a079163-93bb-4c51-aa73-58bda0aa3341 service swift] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:47.071790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1161/2281] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:32:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 09 18:32:59.469710 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005b935-fe43-47e0-8292-5125e7d24496 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:32:59.471157 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-2005b935-fe43-47e0-8292-5125e7d24496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:59.565998 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005b935-fe43-47e0-8292-5125e7d24496 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:32:59.571499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e2396-4a92-41dd-97ac-04e5e13bfd52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f747dcb955fa499a972c46587d3db294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:32:59.572150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e2396-4a92-41dd-97ac-04e5e13bfd52 service placement] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:32:59.572432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e2396-4a92-41dd-97ac-04e5e13bfd52 service placement] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:32:59.572711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e2396-4a92-41dd-97ac-04e5e13bfd52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:32:59.574429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-400e2396-4a92-41dd-97ac-04e5e13bfd52 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:32:59.585430 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e2396-4a92-41dd-97ac-04e5e13bfd52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:32:59.587011 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e2396-4a92-41dd-97ac-04e5e13bfd52 service placement] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:32:59.596672 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1121/2282] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:32:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:33:01.557535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea08645-0f0e-455c-abe1-6f9067291301 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:01.558124 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea08645-0f0e-455c-abe1-6f9067291301 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:01.558124 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea08645-0f0e-455c-abe1-6f9067291301 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:01.559449 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1162/2283] 10.222.0.25 () {54 vars in 921 bytes} [Mon May 9 18:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:33:01.569054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c1b81-29ed-4a90-ab5f-bcdc5cc1e297 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:01.569533 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c1b81-29ed-4a90-ab5f-bcdc5cc1e297 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:01.569664 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c1b81-29ed-4a90-ab5f-bcdc5cc1e297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:01.620202 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-322c1b81-29ed-4a90-ab5f-bcdc5cc1e297 None None] MFA Rules not processed for user `f769ee9f31354e5296645fb1e3bd84b6`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:33:01.621450 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-322c1b81-29ed-4a90-ab5f-bcdc5cc1e297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:33:01.698196 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1122/2284] 10.222.0.25 () {58 vars in 1018 bytes} [Mon May 9 18:33:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:33:01.709671 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea08645-0f0e-455c-abe1-6f9067291301 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:33:01.711144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-6ea08645-0f0e-455c-abe1-6f9067291301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:33:01.784899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea08645-0f0e-455c-abe1-6f9067291301 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:33:01.788313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6bf30-98c6-4f6b-adaf-c54a7978095b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cf284e81ba4042b66c0bc189f1ae2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:33:01.788941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6bf30-98c6-4f6b-adaf-c54a7978095b service glance] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:01.789220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6bf30-98c6-4f6b-adaf-c54a7978095b service glance] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:01.790008 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6bf30-98c6-4f6b-adaf-c54a7978095b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:01.799610 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6bf30-98c6-4f6b-adaf-c54a7978095b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:33:01.801003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6bf30-98c6-4f6b-adaf-c54a7978095b service glance] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:33:01.811774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1163/2285] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:33:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:33:01.879148 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da9a4d-b379-4a42-95d7-7fa7fe983bdd None None] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:01.879460 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da9a4d-b379-4a42-95d7-7fa7fe983bdd None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:01.879746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da9a4d-b379-4a42-95d7-7fa7fe983bdd None None] PATH_INFO: `/` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:01.881147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1123/2286] 10.222.0.25 () {54 vars in 921 bytes} [Mon May 9 18:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:33:01.886019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97ae01-1bac-491a-9bed-ca7565b6f122 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:01.886313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97ae01-1bac-491a-9bed-ca7565b6f122 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:01.886591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97ae01-1bac-491a-9bed-ca7565b6f122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:01.904718 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-ea97ae01-1bac-491a-9bed-ca7565b6f122 None None] MFA Rules not processed for user `6b2b6318776b422d9701351ba6b4ee27`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:33:01.905900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ea97ae01-1bac-491a-9bed-ca7565b6f122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:33:01.954216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1164/2287] 10.222.0.25 () {58 vars in 1018 bytes} [Mon May 9 18:33:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:33:01.966578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da9a4d-b379-4a42-95d7-7fa7fe983bdd None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:33:01.968347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-14da9a4d-b379-4a42-95d7-7fa7fe983bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:33:01.981943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da9a4d-b379-4a42-95d7-7fa7fe983bdd None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:33:01.990756 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccdf14-9a7d-4dda-99fa-975f94bc063f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:33:01.991463 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccdf14-9a7d-4dda-99fa-975f94bc063f service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:01.991743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccdf14-9a7d-4dda-99fa-975f94bc063f service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:01.992014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccdf14-9a7d-4dda-99fa-975f94bc063f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:01.999489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccdf14-9a7d-4dda-99fa-975f94bc063f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:33:02.001284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccdf14-9a7d-4dda-99fa-975f94bc063f service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:33:02.009005 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1124/2288] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:33:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:33:02.090632 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd444f-d3c7-47f3-9e66-fc5b6f600361 None None] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:02.090920 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd444f-d3c7-47f3-9e66-fc5b6f600361 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:02.091232 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd444f-d3c7-47f3-9e66-fc5b6f600361 None None] PATH_INFO: `/` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:02.092639 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1165/2289] 10.222.0.25 () {54 vars in 921 bytes} [Mon May 9 18:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 18:33:02.098029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1d28b-6b96-47e8-9546-5f8a857505e0 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:02.098319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1d28b-6b96-47e8-9546-5f8a857505e0 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:02.098596 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1d28b-6b96-47e8-9546-5f8a857505e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:02.117192 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-94f1d28b-6b96-47e8-9546-5f8a857505e0 None None] MFA Rules not processed for user `f769ee9f31354e5296645fb1e3bd84b6`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:33:02.118382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-94f1d28b-6b96-47e8-9546-5f8a857505e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:33:02.128123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1125/2290] 10.222.0.25 () {58 vars in 1018 bytes} [Mon May 9 18:33:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:33:02.138702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd444f-d3c7-47f3-9e66-fc5b6f600361 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:33:02.141390 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-cfbd444f-d3c7-47f3-9e66-fc5b6f600361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:33:02.164713 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd444f-d3c7-47f3-9e66-fc5b6f600361 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:33:02.168957 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e0006-a62a-4bf2-b1ae-8cefde4134ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:33:02.169602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e0006-a62a-4bf2-b1ae-8cefde4134ea service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:02.169884 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e0006-a62a-4bf2-b1ae-8cefde4134ea service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:02.170160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e0006-a62a-4bf2-b1ae-8cefde4134ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:02.180539 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e0006-a62a-4bf2-b1ae-8cefde4134ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:33:02.181944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e0006-a62a-4bf2-b1ae-8cefde4134ea service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:33:02.192344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1166/2291] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:33:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:33:05.799493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16b009-d5ec-43e5-a33e-976f19b311a1 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:05.799962 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16b009-d5ec-43e5-a33e-976f19b311a1 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:05.800075 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16b009-d5ec-43e5-a33e-976f19b311a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:05.817863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ce16b009-d5ec-43e5-a33e-976f19b311a1 None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:33:05.819343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ce16b009-d5ec-43e5-a33e-976f19b311a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:33:05.828056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1126/2292] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:33:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:33:05.843805 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e0006-a62a-4bf2-b1ae-8cefde4134ea service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:33:05.860743 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e0006-a62a-4bf2-b1ae-8cefde4134ea service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:33:05.866649 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec09ba0-b0d6-4eae-9bcd-95503a3aba2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:33:05.868278 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec09ba0-b0d6-4eae-9bcd-95503a3aba2a service swift] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:05.868748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec09ba0-b0d6-4eae-9bcd-95503a3aba2a service swift] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:05.869101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec09ba0-b0d6-4eae-9bcd-95503a3aba2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:05.881785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec09ba0-b0d6-4eae-9bcd-95503a3aba2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:33:05.884624 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec09ba0-b0d6-4eae-9bcd-95503a3aba2a service swift] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:33:05.895957 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1167/2293] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:33:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 18:33:23.249050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4fb8e-ad78-46d2-9369-bb78e2944cf2 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:23.249926 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4fb8e-ad78-46d2-9369-bb78e2944cf2 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:23.249926 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4fb8e-ad78-46d2-9369-bb78e2944cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:23.277688 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-e3d4fb8e-ad78-46d2-9369-bb78e2944cf2 None None] MFA Rules not processed for user `f99208187e4e4d2eb9f52107ae8f50f2`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:33:23.279015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e3d4fb8e-ad78-46d2-9369-bb78e2944cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:33:23.342591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1127/2294] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:33:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:33:26.763560 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7705b-2215-476e-8efc-dd50344ec801 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:26.764828 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7705b-2215-476e-8efc-dd50344ec801 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:26.765694 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7705b-2215-476e-8efc-dd50344ec801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:26.789152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-fab7705b-2215-476e-8efc-dd50344ec801 None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:33:26.790770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-fab7705b-2215-476e-8efc-dd50344ec801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:33:26.799916 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1168/2295] 10.222.0.25 () {58 vars in 967 bytes} [Mon May 9 18:33:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:33:26.822661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4fb8e-ad78-46d2-9369-bb78e2944cf2 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:33:26.840315 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4fb8e-ad78-46d2-9369-bb78e2944cf2 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:33:26.846487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ace80-8c84-4746-9623-9fcca3ff3fda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:33:26.847081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ace80-8c84-4746-9623-9fcca3ff3fda service swift] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:26.847471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ace80-8c84-4746-9623-9fcca3ff3fda service swift] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:26.847750 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ace80-8c84-4746-9623-9fcca3ff3fda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:26.857968 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ace80-8c84-4746-9623-9fcca3ff3fda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:33:26.859847 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ace80-8c84-4746-9623-9fcca3ff3fda service swift] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:33:26.871879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1128/2296] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:33:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 18:33:51.489293 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c8000-251e-4f33-a544-47465cb8ece5 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:33:51.489830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c8000-251e-4f33-a544-47465cb8ece5 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:33:51.489830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c8000-251e-4f33-a544-47465cb8ece5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:33:51.507409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-d72c8000-251e-4f33-a544-47465cb8ece5 None None] MFA Rules not processed for user `acae3ad19a7e45b9877520dff8f5b995`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:33:51.508565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-d72c8000-251e-4f33-a544-47465cb8ece5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:33:51.560605 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1169/2297] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:33:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:34:16.281452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04280d57-e892-4d8e-9921-8fe0e3991c54 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:34:16.281969 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04280d57-e892-4d8e-9921-8fe0e3991c54 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:34:16.282089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04280d57-e892-4d8e-9921-8fe0e3991c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:34:16.298854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-04280d57-e892-4d8e-9921-8fe0e3991c54 None None] MFA Rules not processed for user `cac32da403af402c8236f46168e81038`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:34:16.300076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-04280d57-e892-4d8e-9921-8fe0e3991c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:34:16.352155 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1129/2298] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:34:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:34:35.944134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c8000-251e-4f33-a544-47465cb8ece5 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:34:35.954100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c8000-251e-4f33-a544-47465cb8ece5 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:34:35.961593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188793ee-fd56-4b3a-81ce-857371946fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:34:35.962359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188793ee-fd56-4b3a-81ce-857371946fad service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:34:35.962773 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188793ee-fd56-4b3a-81ce-857371946fad service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:34:35.963101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188793ee-fd56-4b3a-81ce-857371946fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:34:35.965047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-188793ee-fd56-4b3a-81ce-857371946fad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:34:35.980220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188793ee-fd56-4b3a-81ce-857371946fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:34:35.981785 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188793ee-fd56-4b3a-81ce-857371946fad service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:34:35.996293 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1170/2299] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:34:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:34:36.057072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04280d57-e892-4d8e-9921-8fe0e3991c54 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:34:36.067978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04280d57-e892-4d8e-9921-8fe0e3991c54 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:34:36.071604 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02de19a-4bd5-4f52-ab33-23ae8ac4eb9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:34:36.072248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02de19a-4bd5-4f52-ab33-23ae8ac4eb9a service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:34:36.072530 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02de19a-4bd5-4f52-ab33-23ae8ac4eb9a service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:34:36.072805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02de19a-4bd5-4f52-ab33-23ae8ac4eb9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:34:36.074545 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c02de19a-4bd5-4f52-ab33-23ae8ac4eb9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:34:36.087963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02de19a-4bd5-4f52-ab33-23ae8ac4eb9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:34:36.089379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02de19a-4bd5-4f52-ab33-23ae8ac4eb9a service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:34:36.100300 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1130/2300] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:34:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:34:36.727995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629d8f1-2b5d-43ae-8234-ce2764d2cb08 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:34:36.728327 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629d8f1-2b5d-43ae-8234-ce2764d2cb08 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:34:36.728590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629d8f1-2b5d-43ae-8234-ce2764d2cb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:34:36.745983 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-7629d8f1-2b5d-43ae-8234-ce2764d2cb08 None None] MFA Rules not processed for user `f99208187e4e4d2eb9f52107ae8f50f2`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:34:36.747340 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7629d8f1-2b5d-43ae-8234-ce2764d2cb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:34:36.755991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1171/2301] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:34:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:34:44.113289 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02de19a-4bd5-4f52-ab33-23ae8ac4eb9a service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:34:44.125337 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02de19a-4bd5-4f52-ab33-23ae8ac4eb9a service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:34:44.130423 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced101e8-9038-4468-ad52-5dc3fe407dc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:34:44.131246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced101e8-9038-4468-ad52-5dc3fe407dc3 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:34:44.131851 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced101e8-9038-4468-ad52-5dc3fe407dc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:34:44.132671 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced101e8-9038-4468-ad52-5dc3fe407dc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:34:44.135164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ced101e8-9038-4468-ad52-5dc3fe407dc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:34:44.147719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced101e8-9038-4468-ad52-5dc3fe407dc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:34:44.149516 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced101e8-9038-4468-ad52-5dc3fe407dc3 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:34:44.158832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1131/2302] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:34:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:34:44.561253 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629d8f1-2b5d-43ae-8234-ce2764d2cb08 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:34:44.562642 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7629d8f1-2b5d-43ae-8234-ce2764d2cb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:34:44.578395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629d8f1-2b5d-43ae-8234-ce2764d2cb08 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:34:44.584236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32eae45-aa3c-4a72-b69b-19b2befae2ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f747dcb955fa499a972c46587d3db294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:34:44.585046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32eae45-aa3c-4a72-b69b-19b2befae2ff service placement] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:34:44.585130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32eae45-aa3c-4a72-b69b-19b2befae2ff service placement] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:34:44.585410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32eae45-aa3c-4a72-b69b-19b2befae2ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:34:44.587197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a32eae45-aa3c-4a72-b69b-19b2befae2ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:34:44.599080 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32eae45-aa3c-4a72-b69b-19b2befae2ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:34:44.600702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32eae45-aa3c-4a72-b69b-19b2befae2ff service placement] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:34:44.615638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1172/2303] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:34:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:34:46.377383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced101e8-9038-4468-ad52-5dc3fe407dc3 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:34:46.378827 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ced101e8-9038-4468-ad52-5dc3fe407dc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:34:46.396236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced101e8-9038-4468-ad52-5dc3fe407dc3 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:34:46.401861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670aa52b-895e-48f2-b87d-1f707d939dde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:34:46.402529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aa52b-895e-48f2-b87d-1f707d939dde service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:34:46.402793 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aa52b-895e-48f2-b87d-1f707d939dde service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:34:46.403070 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aa52b-895e-48f2-b87d-1f707d939dde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:34:46.404847 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-670aa52b-895e-48f2-b87d-1f707d939dde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:34:46.415963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670aa52b-895e-48f2-b87d-1f707d939dde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:34:46.417527 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670aa52b-895e-48f2-b87d-1f707d939dde service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:34:46.426597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1132/2304] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:34:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:34:51.395512 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32eae45-aa3c-4a72-b69b-19b2befae2ff service placement] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:34:51.405753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32eae45-aa3c-4a72-b69b-19b2befae2ff service placement] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:34:51.410220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd205a-431f-4562-a754-8a58d02dd4a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:34:51.410873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd205a-431f-4562-a754-8a58d02dd4a9 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:34:51.411154 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd205a-431f-4562-a754-8a58d02dd4a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:34:51.411461 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd205a-431f-4562-a754-8a58d02dd4a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:34:51.413567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-fdcd205a-431f-4562-a754-8a58d02dd4a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:34:51.428813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd205a-431f-4562-a754-8a58d02dd4a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:34:51.429572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd205a-431f-4562-a754-8a58d02dd4a9 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:34:51.438600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1173/2305] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:34:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:34:54.174085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670aa52b-895e-48f2-b87d-1f707d939dde service nova] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:34:54.176118 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-670aa52b-895e-48f2-b87d-1f707d939dde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:34:54.193003 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670aa52b-895e-48f2-b87d-1f707d939dde service nova] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:34:54.201221 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc51527-f6bc-4769-9442-ad2bd1347b52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:34:54.202263 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc51527-f6bc-4769-9442-ad2bd1347b52 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:34:54.202625 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc51527-f6bc-4769-9442-ad2bd1347b52 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:34:54.203199 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc51527-f6bc-4769-9442-ad2bd1347b52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:34:54.205861 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-cfc51527-f6bc-4769-9442-ad2bd1347b52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:34:54.287016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc51527-f6bc-4769-9442-ad2bd1347b52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:34:54.288701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc51527-f6bc-4769-9442-ad2bd1347b52 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:34:54.300979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1133/2306] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:34:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:34:56.516908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e3e5f-e664-40ed-987a-c90657601047 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:34:56.517413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e3e5f-e664-40ed-987a-c90657601047 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:34:56.517524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e3e5f-e664-40ed-987a-c90657601047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:34:56.534163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-064e3e5f-e664-40ed-987a-c90657601047 None None] MFA Rules not processed for user `90fdc96d6e6649a498bb3a8659fcc030`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:34:56.535992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-064e3e5f-e664-40ed-987a-c90657601047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:34:56.542708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1174/2307] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:34:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:34:58.042420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc51527-f6bc-4769-9442-ad2bd1347b52 service nova] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:34:58.054828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc51527-f6bc-4769-9442-ad2bd1347b52 service nova] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:34:58.066559 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32351ed5-f03e-4bfa-9911-9ff043fd4ff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:34:58.067278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351ed5-f03e-4bfa-9911-9ff043fd4ff7 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:34:58.067608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351ed5-f03e-4bfa-9911-9ff043fd4ff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:34:58.067824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351ed5-f03e-4bfa-9911-9ff043fd4ff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:34:58.076794 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-32351ed5-f03e-4bfa-9911-9ff043fd4ff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:34:58.089659 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32351ed5-f03e-4bfa-9911-9ff043fd4ff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:34:58.091223 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32351ed5-f03e-4bfa-9911-9ff043fd4ff7 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:34:58.101863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1134/2308] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:34:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:35:00.278122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e3e5f-e664-40ed-987a-c90657601047 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:00.290832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e3e5f-e664-40ed-987a-c90657601047 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:00.294771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a8dcc-5e32-405e-9b66-2948edbb46c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:00.295749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a8dcc-5e32-405e-9b66-2948edbb46c0 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:00.296230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a8dcc-5e32-405e-9b66-2948edbb46c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:00.296505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a8dcc-5e32-405e-9b66-2948edbb46c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:00.301128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e52a8dcc-5e32-405e-9b66-2948edbb46c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:00.370173 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a8dcc-5e32-405e-9b66-2948edbb46c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:00.371691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a8dcc-5e32-405e-9b66-2948edbb46c0 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:00.383518 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1175/2309] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:35:04.290026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32351ed5-f03e-4bfa-9911-9ff043fd4ff7 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:04.292057 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-32351ed5-f03e-4bfa-9911-9ff043fd4ff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:04.306658 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32351ed5-f03e-4bfa-9911-9ff043fd4ff7 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:04.311122 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66157e3-1445-4255-8e3e-9711e44bcf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:04.312021 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66157e3-1445-4255-8e3e-9711e44bcf3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:04.312301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66157e3-1445-4255-8e3e-9711e44bcf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:04.312573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66157e3-1445-4255-8e3e-9711e44bcf3a admin admin] PATH_INFO: `/v3/users/acae3ad19a7e45b9877520dff8f5b995` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:04.313756 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66157e3-1445-4255-8e3e-9711e44bcf3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=acae3ad19a7e45b9877520dff8f5b995)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:04.315700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66157e3-1445-4255-8e3e-9711e44bcf3a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:04.364534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d66157e3-1445-4255-8e3e-9711e44bcf3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acae3ad19a7e45b9877520dff8f5b995'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:04.371469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d66157e3-1445-4255-8e3e-9711e44bcf3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acae3ad19a7e45b9877520dff8f5b995'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:04.372145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d66157e3-1445-4255-8e3e-9711e44bcf3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acae3ad19a7e45b9877520dff8f5b995'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:04.380088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d66157e3-1445-4255-8e3e-9711e44bcf3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acae3ad19a7e45b9877520dff8f5b995'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:04.381402 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d66157e3-1445-4255-8e3e-9711e44bcf3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acae3ad19a7e45b9877520dff8f5b995'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:04.399070 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1135/2310] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:35:04 2022] DELETE /identity/v3/users/acae3ad19a7e45b9877520dff8f5b995 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:04.565840 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a8dcc-5e32-405e-9b66-2948edbb46c0 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:04.578849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a8dcc-5e32-405e-9b66-2948edbb46c0 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:04.633881 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd1461-be4c-4ba8-8532-087cfecb40b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:04.634735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd1461-be4c-4ba8-8532-087cfecb40b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:04.635019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd1461-be4c-4ba8-8532-087cfecb40b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:04.635322 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd1461-be4c-4ba8-8532-087cfecb40b4 admin admin] PATH_INFO: `/v3/projects/705eecf37fcc4fe7aabef923ece6ad59` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:04.636469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd1461-be4c-4ba8-8532-087cfecb40b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=705eecf37fcc4fe7aabef923ece6ad59)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:04.638331 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd1461-be4c-4ba8-8532-087cfecb40b4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:04.682175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-61bd1461-be4c-4ba8-8532-087cfecb40b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '705eecf37fcc4fe7aabef923ece6ad59', 'request_id': 'req-61bd1461-be4c-4ba8-8532-087cfecb40b4'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:04.687219 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-61bd1461-be4c-4ba8-8532-087cfecb40b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '705eecf37fcc4fe7aabef923ece6ad59', 'request_id': 'req-61bd1461-be4c-4ba8-8532-087cfecb40b4'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:04.694698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-61bd1461-be4c-4ba8-8532-087cfecb40b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '705eecf37fcc4fe7aabef923ece6ad59', 'request_id': 'req-61bd1461-be4c-4ba8-8532-087cfecb40b4'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:04.705166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-61bd1461-be4c-4ba8-8532-087cfecb40b4 admin admin] The token cache is being invalidate because project 705eecf37fcc4fe7aabef923ece6ad59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:35:04.705506 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-61bd1461-be4c-4ba8-8532-087cfecb40b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:04.706693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-61bd1461-be4c-4ba8-8532-087cfecb40b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:04.708237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1176/2311] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:35:04 2022] DELETE /identity/v3/projects/705eecf37fcc4fe7aabef923ece6ad59 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:04.714325 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e53f4-e85f-414a-97fa-953db56c3e7a None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:04.714611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e53f4-e85f-414a-97fa-953db56c3e7a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:04.714892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e53f4-e85f-414a-97fa-953db56c3e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:04.732578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-609e53f4-e85f-414a-97fa-953db56c3e7a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:04.733752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-609e53f4-e85f-414a-97fa-953db56c3e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:04.791791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1136/2312] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:04.799473 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defe54b-48d8-45a4-9efe-69d288bf24c7 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:04.799764 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defe54b-48d8-45a4-9efe-69d288bf24c7 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:04.800043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defe54b-48d8-45a4-9efe-69d288bf24c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:04.818034 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-3defe54b-48d8-45a4-9efe-69d288bf24c7 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:04.819148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3defe54b-48d8-45a4-9efe-69d288bf24c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:04.826849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1177/2313] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:04.831436 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e53f4-e85f-414a-97fa-953db56c3e7a None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:04.841041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e53f4-e85f-414a-97fa-953db56c3e7a None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:04.845537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cad7e-b05c-42ba-8d65-73cba900a8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:04.846185 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cad7e-b05c-42ba-8d65-73cba900a8aa admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:04.846467 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cad7e-b05c-42ba-8d65-73cba900a8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:04.846739 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cad7e-b05c-42ba-8d65-73cba900a8aa admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:04.847496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cad7e-b05c-42ba-8d65-73cba900a8aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:35:04.847803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cad7e-b05c-42ba-8d65-73cba900a8aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:04.849410 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cad7e-b05c-42ba-8d65-73cba900a8aa admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:04.861531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1137/2314] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:35:04.866099 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3defe54b-48d8-45a4-9efe-69d288bf24c7 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:04.875735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3defe54b-48d8-45a4-9efe-69d288bf24c7 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:04.880889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059185a-b7e5-45fc-9a46-31e5a5269bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:04.881535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059185a-b7e5-45fc-9a46-31e5a5269bea admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:04.881814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059185a-b7e5-45fc-9a46-31e5a5269bea admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:04.882088 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059185a-b7e5-45fc-9a46-31e5a5269bea admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:04.882789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059185a-b7e5-45fc-9a46-31e5a5269bea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:04.884708 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059185a-b7e5-45fc-9a46-31e5a5269bea admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:04.909065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1178/2315] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:35:04 2022] POST /identity/v3/projects => generated 409 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:35:04.913212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cad7e-b05c-42ba-8d65-73cba900a8aa admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:04.923483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cad7e-b05c-42ba-8d65-73cba900a8aa admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:04.977478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28fbed-ec99-4db6-8671-a4fe4f6a28d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:04.978155 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28fbed-ec99-4db6-8671-a4fe4f6a28d5 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:04.978439 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28fbed-ec99-4db6-8671-a4fe4f6a28d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:04.978715 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28fbed-ec99-4db6-8671-a4fe4f6a28d5 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:04.979451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28fbed-ec99-4db6-8671-a4fe4f6a28d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:04.981344 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28fbed-ec99-4db6-8671-a4fe4f6a28d5 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:05.004697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1138/2316] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:35:04 2022] POST /identity/v3/users => generated 305 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:35:05.005905 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059185a-b7e5-45fc-9a46-31e5a5269bea admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:05.016257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059185a-b7e5-45fc-9a46-31e5a5269bea admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:05.021884 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c78210-b74d-441c-850c-da391013fa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:05.022567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c78210-b74d-441c-850c-da391013fa3c admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:05.022852 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c78210-b74d-441c-850c-da391013fa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:05.023126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c78210-b74d-441c-850c-da391013fa3c admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:05.023888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c78210-b74d-441c-850c-da391013fa3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:05.025466 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c78210-b74d-441c-850c-da391013fa3c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:05.035875 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1179/2317] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:35:05.039816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28fbed-ec99-4db6-8671-a4fe4f6a28d5 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:05.050460 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28fbed-ec99-4db6-8671-a4fe4f6a28d5 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:05.055105 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d7abc-f1a2-4273-9dfc-5a071eee59db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:05.055798 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d7abc-f1a2-4273-9dfc-5a071eee59db admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:05.056079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d7abc-f1a2-4273-9dfc-5a071eee59db admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:05.056349 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d7abc-f1a2-4273-9dfc-5a071eee59db admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:05.057203 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d7abc-f1a2-4273-9dfc-5a071eee59db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:05.058785 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d7abc-f1a2-4273-9dfc-5a071eee59db admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:05.069374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1139/2318] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:35:05.073598 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c78210-b74d-441c-850c-da391013fa3c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:05.083906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c78210-b74d-441c-850c-da391013fa3c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:05.088593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f774329-30ca-4d45-9278-e0520d845306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:05.089473 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f774329-30ca-4d45-9278-e0520d845306 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:05.089753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f774329-30ca-4d45-9278-e0520d845306 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:05.090026 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f774329-30ca-4d45-9278-e0520d845306 admin admin] PATH_INFO: `/v3/projects/490414b2008e4caf981e00cb5fe971cd/users/edcaa0ec3f93458986b39c1174d6cfc9/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:05.104448 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f774329-30ca-4d45-9278-e0520d845306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=490414b2008e4caf981e00cb5fe971cd, user_id=edcaa0ec3f93458986b39c1174d6cfc9, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:05.107062 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f774329-30ca-4d45-9278-e0520d845306 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:05.122069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1180/2319] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:35:05 2022] PUT /identity/v3/projects/490414b2008e4caf981e00cb5fe971cd/users/edcaa0ec3f93458986b39c1174d6cfc9/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:05.132876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc2956-4407-4372-8d1c-d3c2261a7b26 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:05.133171 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc2956-4407-4372-8d1c-d3c2261a7b26 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:05.133445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc2956-4407-4372-8d1c-d3c2261a7b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:05.161497 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-76dc2956-4407-4372-8d1c-d3c2261a7b26 None None] MFA Rules not processed for user `edcaa0ec3f93458986b39c1174d6cfc9`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:05.162685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-76dc2956-4407-4372-8d1c-d3c2261a7b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:05.240096 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1140/2320] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:05 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:05.244467 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f774329-30ca-4d45-9278-e0520d845306 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:05.305171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f774329-30ca-4d45-9278-e0520d845306 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:05.310052 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da5989-cd15-465f-a32d-35818439e8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:05.310713 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da5989-cd15-465f-a32d-35818439e8bf admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:05.310997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da5989-cd15-465f-a32d-35818439e8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:05.311302 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da5989-cd15-465f-a32d-35818439e8bf admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:05.312017 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da5989-cd15-465f-a32d-35818439e8bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:05.313896 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da5989-cd15-465f-a32d-35818439e8bf admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:05.341857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1181/2321] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:35:05 2022] POST /identity/v3/projects => generated 409 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:35:05.345726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc2956-4407-4372-8d1c-d3c2261a7b26 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:05.356434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc2956-4407-4372-8d1c-d3c2261a7b26 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:05.412152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd01d1e-75be-4340-8b1a-39cdb02fe6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:05.412834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd01d1e-75be-4340-8b1a-39cdb02fe6e7 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:05.413120 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd01d1e-75be-4340-8b1a-39cdb02fe6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:05.413395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd01d1e-75be-4340-8b1a-39cdb02fe6e7 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:05.414072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd01d1e-75be-4340-8b1a-39cdb02fe6e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:05.416001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd01d1e-75be-4340-8b1a-39cdb02fe6e7 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:05.439293 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1141/2322] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:35:05 2022] POST /identity/v3/users => generated 311 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:35:05.443065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da5989-cd15-465f-a32d-35818439e8bf admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:05.454028 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da5989-cd15-465f-a32d-35818439e8bf admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:05.459018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e5415-b377-4e09-8158-f2230f6fcb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:05.459703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e5415-b377-4e09-8158-f2230f6fcb40 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:05.460011 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e5415-b377-4e09-8158-f2230f6fcb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:05.460283 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e5415-b377-4e09-8158-f2230f6fcb40 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:05.461023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e5415-b377-4e09-8158-f2230f6fcb40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:05.462633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e5415-b377-4e09-8158-f2230f6fcb40 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:05.474482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1182/2323] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:35:05.478914 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd01d1e-75be-4340-8b1a-39cdb02fe6e7 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:05.489448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd01d1e-75be-4340-8b1a-39cdb02fe6e7 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:05.494041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96490310-eb38-494e-8faa-3226db181203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:05.494955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96490310-eb38-494e-8faa-3226db181203 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:05.495295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96490310-eb38-494e-8faa-3226db181203 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:05.495574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96490310-eb38-494e-8faa-3226db181203 admin admin] PATH_INFO: `/v3/projects/475f6bda500f45cab1f8a5784139a235/users/b3942b1fbb0d4afdbcdc3323f24c023a/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:05.510661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96490310-eb38-494e-8faa-3226db181203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=475f6bda500f45cab1f8a5784139a235, user_id=b3942b1fbb0d4afdbcdc3323f24c023a, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:05.513311 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96490310-eb38-494e-8faa-3226db181203 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:05.529449 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1142/2324] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:35:05 2022] PUT /identity/v3/projects/475f6bda500f45cab1f8a5784139a235/users/b3942b1fbb0d4afdbcdc3323f24c023a/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:35:05.533467 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e5415-b377-4e09-8158-f2230f6fcb40 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:05.551478 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e5415-b377-4e09-8158-f2230f6fcb40 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:05.608081 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed990166-fd47-45f8-bd66-056d84e3f4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:05.608751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed990166-fd47-45f8-bd66-056d84e3f4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:05.609033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed990166-fd47-45f8-bd66-056d84e3f4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:05.609305 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed990166-fd47-45f8-bd66-056d84e3f4d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:05.610012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed990166-fd47-45f8-bd66-056d84e3f4d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:05.611618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed990166-fd47-45f8-bd66-056d84e3f4d2 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:05.621667 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1183/2325] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:35:05.626065 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96490310-eb38-494e-8faa-3226db181203 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:05.636016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96490310-eb38-494e-8faa-3226db181203 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:05.640703 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cabca-0072-42d9-99fa-a21e546fa03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:05.641574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cabca-0072-42d9-99fa-a21e546fa03f admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:05.641848 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cabca-0072-42d9-99fa-a21e546fa03f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:05.642122 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cabca-0072-42d9-99fa-a21e546fa03f admin admin] PATH_INFO: `/v3/projects/475f6bda500f45cab1f8a5784139a235/users/b3942b1fbb0d4afdbcdc3323f24c023a/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:05.644346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cabca-0072-42d9-99fa-a21e546fa03f admin admin] RBAC: Authorizing `identity:create_grant(project_id=475f6bda500f45cab1f8a5784139a235, user_id=b3942b1fbb0d4afdbcdc3323f24c023a, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:05.647138 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cabca-0072-42d9-99fa-a21e546fa03f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:05.658249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1143/2326] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:35:05 2022] PUT /identity/v3/projects/475f6bda500f45cab1f8a5784139a235/users/b3942b1fbb0d4afdbcdc3323f24c023a/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:05.669850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c271e-5b34-46b1-9ff4-1dc968297a44 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:05.670136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c271e-5b34-46b1-9ff4-1dc968297a44 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:05.670414 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c271e-5b34-46b1-9ff4-1dc968297a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:05.697478 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-808c271e-5b34-46b1-9ff4-1dc968297a44 None None] MFA Rules not processed for user `b3942b1fbb0d4afdbcdc3323f24c023a`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:05.698676 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-808c271e-5b34-46b1-9ff4-1dc968297a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:05.762971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1184/2327] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:05 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:05.773528 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cabca-0072-42d9-99fa-a21e546fa03f admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:05.775357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-824cabca-0072-42d9-99fa-a21e546fa03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:05.861131 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cabca-0072-42d9-99fa-a21e546fa03f admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:05.865117 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde348f-dd31-472f-97bd-94baa3305cac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:05.865659 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde348f-dd31-472f-97bd-94baa3305cac service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:05.865939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde348f-dd31-472f-97bd-94baa3305cac service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:05.866218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde348f-dd31-472f-97bd-94baa3305cac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:05.875719 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde348f-dd31-472f-97bd-94baa3305cac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:05.877134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde348f-dd31-472f-97bd-94baa3305cac service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:05.886379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1144/2328] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:05 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:35:06.029612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c271e-5b34-46b1-9ff4-1dc968297a44 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:06.039413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c271e-5b34-46b1-9ff4-1dc968297a44 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:06.042897 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec16e39-a8db-40d1-9c23-3d7fa7c803f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:06.043552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec16e39-a8db-40d1-9c23-3d7fa7c803f9 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:06.043830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec16e39-a8db-40d1-9c23-3d7fa7c803f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:06.044101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec16e39-a8db-40d1-9c23-3d7fa7c803f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:06.097486 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec16e39-a8db-40d1-9c23-3d7fa7c803f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:06.098883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec16e39-a8db-40d1-9c23-3d7fa7c803f9 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:06.107409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1185/2329] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:06 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:35:09.194377 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde348f-dd31-472f-97bd-94baa3305cac service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:09.195808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ccde348f-dd31-472f-97bd-94baa3305cac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:09.268619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde348f-dd31-472f-97bd-94baa3305cac service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:09.274040 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f11a0-81ad-44bc-b521-0fb84e8c8c50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:09.274810 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f11a0-81ad-44bc-b521-0fb84e8c8c50 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:09.275094 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f11a0-81ad-44bc-b521-0fb84e8c8c50 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:09.275398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f11a0-81ad-44bc-b521-0fb84e8c8c50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:09.277149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e01f11a0-81ad-44bc-b521-0fb84e8c8c50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:09.289495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f11a0-81ad-44bc-b521-0fb84e8c8c50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:09.291053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f11a0-81ad-44bc-b521-0fb84e8c8c50 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:09.301822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1145/2330] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:35:13.336077 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec16e39-a8db-40d1-9c23-3d7fa7c803f9 service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:13.395461 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec16e39-a8db-40d1-9c23-3d7fa7c803f9 service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:13.399859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf1708-31cb-4749-b048-2c6fc5087fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:13.400721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf1708-31cb-4749-b048-2c6fc5087fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:13.400998 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf1708-31cb-4749-b048-2c6fc5087fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:13.401269 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf1708-31cb-4749-b048-2c6fc5087fe6 admin admin] PATH_INFO: `/v3/users/edcaa0ec3f93458986b39c1174d6cfc9` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:13.402844 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf1708-31cb-4749-b048-2c6fc5087fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edcaa0ec3f93458986b39c1174d6cfc9)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:13.405084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf1708-31cb-4749-b048-2c6fc5087fe6 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:13.456472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e4bf1708-31cb-4749-b048-2c6fc5087fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edcaa0ec3f93458986b39c1174d6cfc9'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:13.457129 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e4bf1708-31cb-4749-b048-2c6fc5087fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edcaa0ec3f93458986b39c1174d6cfc9'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:13.465856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e4bf1708-31cb-4749-b048-2c6fc5087fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edcaa0ec3f93458986b39c1174d6cfc9'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:13.474703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e4bf1708-31cb-4749-b048-2c6fc5087fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edcaa0ec3f93458986b39c1174d6cfc9'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:13.475440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e4bf1708-31cb-4749-b048-2c6fc5087fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edcaa0ec3f93458986b39c1174d6cfc9'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:13.492890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1186/2331] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:35:13 2022] DELETE /identity/v3/users/edcaa0ec3f93458986b39c1174d6cfc9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:13.497720 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924914c-e90d-4d30-8bcc-909e313a4aec None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:13.498276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924914c-e90d-4d30-8bcc-909e313a4aec None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:13.498276 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924914c-e90d-4d30-8bcc-909e313a4aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:13.515533 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-a924914c-e90d-4d30-8bcc-909e313a4aec None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:13.517546 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a924914c-e90d-4d30-8bcc-909e313a4aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:13.576504 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1146/2332] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:13.589116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf1708-31cb-4749-b048-2c6fc5087fe6 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:13.590257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e4bf1708-31cb-4749-b048-2c6fc5087fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:13.651645 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf1708-31cb-4749-b048-2c6fc5087fe6 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:13.655313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43771b-c6fc-4249-941d-5df4c8fc0438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:13.655965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43771b-c6fc-4249-941d-5df4c8fc0438 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:13.656249 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43771b-c6fc-4249-941d-5df4c8fc0438 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:13.656528 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43771b-c6fc-4249-941d-5df4c8fc0438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:13.665935 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43771b-c6fc-4249-941d-5df4c8fc0438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:13.667381 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43771b-c6fc-4249-941d-5df4c8fc0438 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:13.676582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1187/2333] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:35:13.930970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924914c-e90d-4d30-8bcc-909e313a4aec None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:13.932488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a924914c-e90d-4d30-8bcc-909e313a4aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:13.945184 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924914c-e90d-4d30-8bcc-909e313a4aec None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:13.949764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:13.950564 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:13.950847 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:13.951120 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] PATH_INFO: `/v3/projects/490414b2008e4caf981e00cb5fe971cd` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:13.952346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=490414b2008e4caf981e00cb5fe971cd)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:13.954295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:14.000498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '490414b2008e4caf981e00cb5fe971cd', 'request_id': 'req-3e993595-a471-48d2-a047-f1733d3d7b82'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.006260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '490414b2008e4caf981e00cb5fe971cd', 'request_id': 'req-3e993595-a471-48d2-a047-f1733d3d7b82'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.013313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '490414b2008e4caf981e00cb5fe971cd', 'request_id': 'req-3e993595-a471-48d2-a047-f1733d3d7b82'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.024801 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] The token cache is being invalidate because project 490414b2008e4caf981e00cb5fe971cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:35:14.025089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.025716 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.027792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1147/2334] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:35:13 2022] DELETE /identity/v3/projects/490414b2008e4caf981e00cb5fe971cd => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:14.031775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43771b-c6fc-4249-941d-5df4c8fc0438 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:14.041142 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43771b-c6fc-4249-941d-5df4c8fc0438 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:14.094773 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdff62a-dbe5-4f9d-b102-16920d130c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:14.095691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdff62a-dbe5-4f9d-b102-16920d130c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:14.096001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdff62a-dbe5-4f9d-b102-16920d130c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:14.096274 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdff62a-dbe5-4f9d-b102-16920d130c99 admin admin] PATH_INFO: `/v3/users/b3942b1fbb0d4afdbcdc3323f24c023a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:14.097438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdff62a-dbe5-4f9d-b102-16920d130c99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3942b1fbb0d4afdbcdc3323f24c023a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:14.099364 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdff62a-dbe5-4f9d-b102-16920d130c99 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:14.143626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5fdff62a-dbe5-4f9d-b102-16920d130c99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3942b1fbb0d4afdbcdc3323f24c023a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.144273 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5fdff62a-dbe5-4f9d-b102-16920d130c99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3942b1fbb0d4afdbcdc3323f24c023a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.152892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5fdff62a-dbe5-4f9d-b102-16920d130c99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3942b1fbb0d4afdbcdc3323f24c023a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.160360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5fdff62a-dbe5-4f9d-b102-16920d130c99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3942b1fbb0d4afdbcdc3323f24c023a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.161011 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5fdff62a-dbe5-4f9d-b102-16920d130c99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3942b1fbb0d4afdbcdc3323f24c023a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.177903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1188/2335] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:35:14 2022] DELETE /identity/v3/users/b3942b1fbb0d4afdbcdc3323f24c023a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:14.432259 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:14.433692 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:14.449434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e993595-a471-48d2-a047-f1733d3d7b82 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:14.506937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70bbf4-a345-497e-91ff-9b29699a0f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:14.507832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70bbf4-a345-497e-91ff-9b29699a0f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:14.508117 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70bbf4-a345-497e-91ff-9b29699a0f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:14.508394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70bbf4-a345-497e-91ff-9b29699a0f4c admin admin] PATH_INFO: `/v3/projects/475f6bda500f45cab1f8a5784139a235` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:14.509599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70bbf4-a345-497e-91ff-9b29699a0f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=475f6bda500f45cab1f8a5784139a235)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:14.511536 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70bbf4-a345-497e-91ff-9b29699a0f4c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:14.557859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4c70bbf4-a345-497e-91ff-9b29699a0f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '475f6bda500f45cab1f8a5784139a235', 'request_id': 'req-4c70bbf4-a345-497e-91ff-9b29699a0f4c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.562867 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4c70bbf4-a345-497e-91ff-9b29699a0f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '475f6bda500f45cab1f8a5784139a235', 'request_id': 'req-4c70bbf4-a345-497e-91ff-9b29699a0f4c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.569708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4c70bbf4-a345-497e-91ff-9b29699a0f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '475f6bda500f45cab1f8a5784139a235', 'request_id': 'req-4c70bbf4-a345-497e-91ff-9b29699a0f4c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.580382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4c70bbf4-a345-497e-91ff-9b29699a0f4c admin admin] The token cache is being invalidate because project 475f6bda500f45cab1f8a5784139a235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:35:14.580669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4c70bbf4-a345-497e-91ff-9b29699a0f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.581299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4c70bbf4-a345-497e-91ff-9b29699a0f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:14.582821 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1148/2336] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:35:14 2022] DELETE /identity/v3/projects/475f6bda500f45cab1f8a5784139a235 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:14.587719 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeaafc5-4807-44a5-a62e-a04708eae079 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:14.588173 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeaafc5-4807-44a5-a62e-a04708eae079 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:14.588283 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeaafc5-4807-44a5-a62e-a04708eae079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:14.606139 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-cbeaafc5-4807-44a5-a62e-a04708eae079 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:14.607345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-cbeaafc5-4807-44a5-a62e-a04708eae079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:14.665492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1189/2337] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:14.673428 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100947f-55ca-422c-9ac1-84cad842431b None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:14.673716 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100947f-55ca-422c-9ac1-84cad842431b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:14.674055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100947f-55ca-422c-9ac1-84cad842431b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:14.692809 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-0100947f-55ca-422c-9ac1-84cad842431b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:14.694008 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-0100947f-55ca-422c-9ac1-84cad842431b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:14.704375 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1149/2338] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:14.706268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeaafc5-4807-44a5-a62e-a04708eae079 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:14.715803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeaafc5-4807-44a5-a62e-a04708eae079 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:14.721908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bc090-bbbc-4ad5-9e51-74db087c2e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:14.722156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bc090-bbbc-4ad5-9e51-74db087c2e99 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:14.722260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bc090-bbbc-4ad5-9e51-74db087c2e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:14.722483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bc090-bbbc-4ad5-9e51-74db087c2e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:14.723224 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bc090-bbbc-4ad5-9e51-74db087c2e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:35:14.723593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bc090-bbbc-4ad5-9e51-74db087c2e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:14.725181 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bc090-bbbc-4ad5-9e51-74db087c2e99 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:14.737896 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1190/2339] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:35:14.742311 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100947f-55ca-422c-9ac1-84cad842431b None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:14.751885 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100947f-55ca-422c-9ac1-84cad842431b None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:14.756526 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003ff9b-ce1f-4158-b4ef-e7da76d149ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:14.757180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003ff9b-ce1f-4158-b4ef-e7da76d149ce admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:14.757461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003ff9b-ce1f-4158-b4ef-e7da76d149ce admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:14.757799 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003ff9b-ce1f-4158-b4ef-e7da76d149ce admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:14.758732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003ff9b-ce1f-4158-b4ef-e7da76d149ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:14.760755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003ff9b-ce1f-4158-b4ef-e7da76d149ce admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:14.786996 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1150/2340] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:35:14 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:35:14.792245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bc090-bbbc-4ad5-9e51-74db087c2e99 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:14.802630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bc090-bbbc-4ad5-9e51-74db087c2e99 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:14.858087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b183f9-1068-4156-863d-7e36baa69a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:14.858673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b183f9-1068-4156-863d-7e36baa69a70 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:14.858949 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b183f9-1068-4156-863d-7e36baa69a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:14.859246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b183f9-1068-4156-863d-7e36baa69a70 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:14.859916 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b183f9-1068-4156-863d-7e36baa69a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:14.861855 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b183f9-1068-4156-863d-7e36baa69a70 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:14.881936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1191/2341] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:35:14 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:35:14.885966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003ff9b-ce1f-4158-b4ef-e7da76d149ce admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:14.896591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003ff9b-ce1f-4158-b4ef-e7da76d149ce admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:14.901100 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd049f65-1848-4b3f-8d70-a14600800599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:14.901749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd049f65-1848-4b3f-8d70-a14600800599 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:14.902054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd049f65-1848-4b3f-8d70-a14600800599 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:14.902326 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd049f65-1848-4b3f-8d70-a14600800599 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:14.903037 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd049f65-1848-4b3f-8d70-a14600800599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:14.904637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd049f65-1848-4b3f-8d70-a14600800599 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:14.915259 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1151/2342] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:35:14.918886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b183f9-1068-4156-863d-7e36baa69a70 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:14.928326 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b183f9-1068-4156-863d-7e36baa69a70 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:14.933302 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62583c55-118a-4c5e-8954-d314c494f5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:14.933925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62583c55-118a-4c5e-8954-d314c494f5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:14.934203 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62583c55-118a-4c5e-8954-d314c494f5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:14.934472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62583c55-118a-4c5e-8954-d314c494f5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:14.935205 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62583c55-118a-4c5e-8954-d314c494f5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:14.936766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62583c55-118a-4c5e-8954-d314c494f5a2 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:14.946143 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1192/2343] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:35:14.950132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd049f65-1848-4b3f-8d70-a14600800599 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:14.960356 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd049f65-1848-4b3f-8d70-a14600800599 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:14.965014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6a66a-baed-422e-b894-279618196845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:14.965881 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6a66a-baed-422e-b894-279618196845 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:14.966160 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6a66a-baed-422e-b894-279618196845 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:14.966432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6a66a-baed-422e-b894-279618196845 admin admin] PATH_INFO: `/v3/projects/d3bc93b00fd5443d833ba443656307fa/users/b3519945f75a427e999925d1545cc9c6/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:14.981319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6a66a-baed-422e-b894-279618196845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3bc93b00fd5443d833ba443656307fa, user_id=b3519945f75a427e999925d1545cc9c6, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:14.983978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6a66a-baed-422e-b894-279618196845 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:14.996489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1152/2344] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:35:14 2022] PUT /identity/v3/projects/d3bc93b00fd5443d833ba443656307fa/users/b3519945f75a427e999925d1545cc9c6/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:15.007023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3318a1a-09b6-4d94-833d-b56ba1fd3b17 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:15.007335 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3318a1a-09b6-4d94-833d-b56ba1fd3b17 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:15.007608 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3318a1a-09b6-4d94-833d-b56ba1fd3b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:15.034529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-d3318a1a-09b6-4d94-833d-b56ba1fd3b17 None None] MFA Rules not processed for user `b3519945f75a427e999925d1545cc9c6`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:15.035769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-d3318a1a-09b6-4d94-833d-b56ba1fd3b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:15.095494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1193/2345] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:15.100470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6a66a-baed-422e-b894-279618196845 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:15.159714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6a66a-baed-422e-b894-279618196845 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:15.164215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45375a-3ecc-49b0-baa4-fdaf961655ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:15.164858 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45375a-3ecc-49b0-baa4-fdaf961655ae admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:15.165138 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45375a-3ecc-49b0-baa4-fdaf961655ae admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:15.165410 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45375a-3ecc-49b0-baa4-fdaf961655ae admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:15.166099 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45375a-3ecc-49b0-baa4-fdaf961655ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:15.168034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45375a-3ecc-49b0-baa4-fdaf961655ae admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:15.192555 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1153/2346] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:35:15 2022] POST /identity/v3/projects => generated 379 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:35:15.196560 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3318a1a-09b6-4d94-833d-b56ba1fd3b17 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:15.207574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3318a1a-09b6-4d94-833d-b56ba1fd3b17 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:15.260261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c3338-7acc-40bb-838f-851a21f7b822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:15.260928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c3338-7acc-40bb-838f-851a21f7b822 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:15.261210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c3338-7acc-40bb-838f-851a21f7b822 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:15.261489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c3338-7acc-40bb-838f-851a21f7b822 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:15.262279 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c3338-7acc-40bb-838f-851a21f7b822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:15.264096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c3338-7acc-40bb-838f-851a21f7b822 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:15.283206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1194/2347] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:35:15 2022] POST /identity/v3/users => generated 296 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:35:15.288116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45375a-3ecc-49b0-baa4-fdaf961655ae admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:15.298321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45375a-3ecc-49b0-baa4-fdaf961655ae admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:15.302914 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f46eed-1f1f-4d47-b850-02408541c19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:15.303600 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f46eed-1f1f-4d47-b850-02408541c19a admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:15.303891 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f46eed-1f1f-4d47-b850-02408541c19a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:15.304170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f46eed-1f1f-4d47-b850-02408541c19a admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:15.304888 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f46eed-1f1f-4d47-b850-02408541c19a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:15.306525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f46eed-1f1f-4d47-b850-02408541c19a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:15.317181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1154/2348] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:35:15.321048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c3338-7acc-40bb-838f-851a21f7b822 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:15.331328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c3338-7acc-40bb-838f-851a21f7b822 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:15.335925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a72f4-3024-4210-8e74-ae23a9f38036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:15.336791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a72f4-3024-4210-8e74-ae23a9f38036 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:15.337075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a72f4-3024-4210-8e74-ae23a9f38036 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:15.337354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a72f4-3024-4210-8e74-ae23a9f38036 admin admin] PATH_INFO: `/v3/projects/5df6f77658c04ebc8c702ee25563e607/users/e52418ec07e04773a4308f0a4f9270eb/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:15.351665 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a72f4-3024-4210-8e74-ae23a9f38036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df6f77658c04ebc8c702ee25563e607, user_id=e52418ec07e04773a4308f0a4f9270eb, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:15.354270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a72f4-3024-4210-8e74-ae23a9f38036 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:15.366596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1195/2349] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:35:15 2022] PUT /identity/v3/projects/5df6f77658c04ebc8c702ee25563e607/users/e52418ec07e04773a4308f0a4f9270eb/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:15.370531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f46eed-1f1f-4d47-b850-02408541c19a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:15.381181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f46eed-1f1f-4d47-b850-02408541c19a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:15.434629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e400c-08e1-4eab-be3f-9c8f2b647013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:15.435243 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e400c-08e1-4eab-be3f-9c8f2b647013 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:15.435532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e400c-08e1-4eab-be3f-9c8f2b647013 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:15.435811 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e400c-08e1-4eab-be3f-9c8f2b647013 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:15.436536 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e400c-08e1-4eab-be3f-9c8f2b647013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:15.438112 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e400c-08e1-4eab-be3f-9c8f2b647013 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:15.448172 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1155/2350] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:35:15.452311 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a72f4-3024-4210-8e74-ae23a9f38036 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:15.462818 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a72f4-3024-4210-8e74-ae23a9f38036 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:15.467372 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304cd9a-fa61-4cb1-89b2-3d78e6005903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:15.468240 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304cd9a-fa61-4cb1-89b2-3d78e6005903 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:15.468519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304cd9a-fa61-4cb1-89b2-3d78e6005903 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:15.468797 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304cd9a-fa61-4cb1-89b2-3d78e6005903 admin admin] PATH_INFO: `/v3/projects/5df6f77658c04ebc8c702ee25563e607/users/e52418ec07e04773a4308f0a4f9270eb/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:15.470744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304cd9a-fa61-4cb1-89b2-3d78e6005903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df6f77658c04ebc8c702ee25563e607, user_id=e52418ec07e04773a4308f0a4f9270eb, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:15.473450 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304cd9a-fa61-4cb1-89b2-3d78e6005903 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:15.483591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1196/2351] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:35:15 2022] PUT /identity/v3/projects/5df6f77658c04ebc8c702ee25563e607/users/e52418ec07e04773a4308f0a4f9270eb/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:15.494623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc535249-eb0e-485f-b766-bbdd369d2285 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:15.494917 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc535249-eb0e-485f-b766-bbdd369d2285 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:15.495237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc535249-eb0e-485f-b766-bbdd369d2285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:15.521783 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-bc535249-eb0e-485f-b766-bbdd369d2285 None None] MFA Rules not processed for user `e52418ec07e04773a4308f0a4f9270eb`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:15.522972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-bc535249-eb0e-485f-b766-bbdd369d2285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:15.592618 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1156/2352] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:15.603376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304cd9a-fa61-4cb1-89b2-3d78e6005903 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:15.605131 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2304cd9a-fa61-4cb1-89b2-3d78e6005903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:15.664133 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304cd9a-fa61-4cb1-89b2-3d78e6005903 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:15.667712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b490d2-e27f-44bf-83d3-2f49a17332e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:15.668348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b490d2-e27f-44bf-83d3-2f49a17332e5 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:15.668633 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b490d2-e27f-44bf-83d3-2f49a17332e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:15.668904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b490d2-e27f-44bf-83d3-2f49a17332e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:15.678738 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b490d2-e27f-44bf-83d3-2f49a17332e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:15.680183 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b490d2-e27f-44bf-83d3-2f49a17332e5 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:15.692856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1197/2353] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:35:16.079209 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc535249-eb0e-485f-b766-bbdd369d2285 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:16.139313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc535249-eb0e-485f-b766-bbdd369d2285 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:16.143851 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38feae8d-b5ce-4066-b73c-8657f46f9e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:16.144727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feae8d-b5ce-4066-b73c-8657f46f9e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:16.145179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feae8d-b5ce-4066-b73c-8657f46f9e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:16.145458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feae8d-b5ce-4066-b73c-8657f46f9e01 admin admin] PATH_INFO: `/v3/users/b3519945f75a427e999925d1545cc9c6` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:16.146671 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38feae8d-b5ce-4066-b73c-8657f46f9e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3519945f75a427e999925d1545cc9c6)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:16.148637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38feae8d-b5ce-4066-b73c-8657f46f9e01 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:16.192947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-38feae8d-b5ce-4066-b73c-8657f46f9e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3519945f75a427e999925d1545cc9c6'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.199578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-38feae8d-b5ce-4066-b73c-8657f46f9e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3519945f75a427e999925d1545cc9c6'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.200194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-38feae8d-b5ce-4066-b73c-8657f46f9e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3519945f75a427e999925d1545cc9c6'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.206806 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-38feae8d-b5ce-4066-b73c-8657f46f9e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3519945f75a427e999925d1545cc9c6'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.207462 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-38feae8d-b5ce-4066-b73c-8657f46f9e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3519945f75a427e999925d1545cc9c6'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.224556 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1157/2354] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:35:16 2022] DELETE /identity/v3/users/b3519945f75a427e999925d1545cc9c6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:16.229456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3815f-7f1b-47ed-9ca9-e2aa2ab6bbc9 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:16.229740 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3815f-7f1b-47ed-9ca9-e2aa2ab6bbc9 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:16.230042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3815f-7f1b-47ed-9ca9-e2aa2ab6bbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:16.248121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4af3815f-7f1b-47ed-9ca9-e2aa2ab6bbc9 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:16.249366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4af3815f-7f1b-47ed-9ca9-e2aa2ab6bbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:16.306493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1198/2355] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:16.316139 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38feae8d-b5ce-4066-b73c-8657f46f9e01 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:16.317529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-38feae8d-b5ce-4066-b73c-8657f46f9e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:16.372150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38feae8d-b5ce-4066-b73c-8657f46f9e01 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:16.376014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4a1b7-7999-40f9-b105-745fdcdf6c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:16.376662 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4a1b7-7999-40f9-b105-745fdcdf6c8b service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:16.376945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4a1b7-7999-40f9-b105-745fdcdf6c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:16.377258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4a1b7-7999-40f9-b105-745fdcdf6c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:16.386056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4a1b7-7999-40f9-b105-745fdcdf6c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:16.387486 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4a1b7-7999-40f9-b105-745fdcdf6c8b service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:16.397186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1158/2356] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:35:16.656126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3815f-7f1b-47ed-9ca9-e2aa2ab6bbc9 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:16.657545 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4af3815f-7f1b-47ed-9ca9-e2aa2ab6bbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:16.670073 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3815f-7f1b-47ed-9ca9-e2aa2ab6bbc9 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:16.674706 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:16.675595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:16.675873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:16.676140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] PATH_INFO: `/v3/projects/d3bc93b00fd5443d833ba443656307fa` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:16.677309 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3bc93b00fd5443d833ba443656307fa)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:16.679277 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:16.724006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3bc93b00fd5443d833ba443656307fa', 'request_id': 'req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.729637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3bc93b00fd5443d833ba443656307fa', 'request_id': 'req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.736079 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3bc93b00fd5443d833ba443656307fa', 'request_id': 'req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.746687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] The token cache is being invalidate because project d3bc93b00fd5443d833ba443656307fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:35:16.746964 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.747848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.749337 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1199/2357] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:35:16 2022] DELETE /identity/v3/projects/d3bc93b00fd5443d833ba443656307fa => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:35:16.754006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4a1b7-7999-40f9-b105-745fdcdf6c8b service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:16.763453 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4a1b7-7999-40f9-b105-745fdcdf6c8b service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:16.817577 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8dbcad-65ff-4412-baf2-14313400ac8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:16.818433 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8dbcad-65ff-4412-baf2-14313400ac8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:16.818711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8dbcad-65ff-4412-baf2-14313400ac8f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:16.818986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8dbcad-65ff-4412-baf2-14313400ac8f admin admin] PATH_INFO: `/v3/users/e52418ec07e04773a4308f0a4f9270eb` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:16.820179 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8dbcad-65ff-4412-baf2-14313400ac8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e52418ec07e04773a4308f0a4f9270eb)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:16.822068 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8dbcad-65ff-4412-baf2-14313400ac8f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:16.865607 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ad8dbcad-65ff-4412-baf2-14313400ac8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e52418ec07e04773a4308f0a4f9270eb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.872188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ad8dbcad-65ff-4412-baf2-14313400ac8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e52418ec07e04773a4308f0a4f9270eb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.873090 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ad8dbcad-65ff-4412-baf2-14313400ac8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e52418ec07e04773a4308f0a4f9270eb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.880091 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ad8dbcad-65ff-4412-baf2-14313400ac8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e52418ec07e04773a4308f0a4f9270eb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.880691 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ad8dbcad-65ff-4412-baf2-14313400ac8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e52418ec07e04773a4308f0a4f9270eb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:16.897070 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1159/2358] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:35:16 2022] DELETE /identity/v3/users/e52418ec07e04773a4308f0a4f9270eb => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:17.162816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:17.164272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:17.176744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b296ee5-e2dd-4167-8c6e-e90c0ffa9e59 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:17.230925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0a3c9-2c77-4249-869b-a061b3e0120a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:17.231815 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0a3c9-2c77-4249-869b-a061b3e0120a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:17.232093 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0a3c9-2c77-4249-869b-a061b3e0120a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:17.232364 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0a3c9-2c77-4249-869b-a061b3e0120a admin admin] PATH_INFO: `/v3/projects/5df6f77658c04ebc8c702ee25563e607` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:17.233501 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0a3c9-2c77-4249-869b-a061b3e0120a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df6f77658c04ebc8c702ee25563e607)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:17.235674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0a3c9-2c77-4249-869b-a061b3e0120a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:17.279992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e3d0a3c9-2c77-4249-869b-a061b3e0120a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df6f77658c04ebc8c702ee25563e607', 'request_id': 'req-e3d0a3c9-2c77-4249-869b-a061b3e0120a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:17.285803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e3d0a3c9-2c77-4249-869b-a061b3e0120a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df6f77658c04ebc8c702ee25563e607', 'request_id': 'req-e3d0a3c9-2c77-4249-869b-a061b3e0120a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:17.292651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e3d0a3c9-2c77-4249-869b-a061b3e0120a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df6f77658c04ebc8c702ee25563e607', 'request_id': 'req-e3d0a3c9-2c77-4249-869b-a061b3e0120a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:17.304069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e3d0a3c9-2c77-4249-869b-a061b3e0120a admin admin] The token cache is being invalidate because project 5df6f77658c04ebc8c702ee25563e607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:35:17.304354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e3d0a3c9-2c77-4249-869b-a061b3e0120a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:17.304974 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e3d0a3c9-2c77-4249-869b-a061b3e0120a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:17.306482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1200/2359] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:35:17 2022] DELETE /identity/v3/projects/5df6f77658c04ebc8c702ee25563e607 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:17.312114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446de47e-41c1-4973-9c2a-111e4a16fee3 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:17.312402 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446de47e-41c1-4973-9c2a-111e4a16fee3 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:17.312680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446de47e-41c1-4973-9c2a-111e4a16fee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:17.330993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-446de47e-41c1-4973-9c2a-111e4a16fee3 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:17.332287 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-446de47e-41c1-4973-9c2a-111e4a16fee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:17.388613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1160/2360] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:17.396831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436c3d4-c8a4-4a7e-b8ff-36566c59065c None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:17.397115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436c3d4-c8a4-4a7e-b8ff-36566c59065c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:17.397389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436c3d4-c8a4-4a7e-b8ff-36566c59065c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:17.415540 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-f436c3d4-c8a4-4a7e-b8ff-36566c59065c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:17.416707 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f436c3d4-c8a4-4a7e-b8ff-36566c59065c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:17.424456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1201/2361] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:17.428334 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446de47e-41c1-4973-9c2a-111e4a16fee3 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:17.437730 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446de47e-41c1-4973-9c2a-111e4a16fee3 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:17.442846 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56659a53-2f48-4d59-bba2-5146f307878f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:17.443533 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56659a53-2f48-4d59-bba2-5146f307878f admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:17.443818 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56659a53-2f48-4d59-bba2-5146f307878f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:17.444089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56659a53-2f48-4d59-bba2-5146f307878f admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:17.444807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56659a53-2f48-4d59-bba2-5146f307878f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:35:17.445111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56659a53-2f48-4d59-bba2-5146f307878f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:17.446685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56659a53-2f48-4d59-bba2-5146f307878f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:17.458516 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1161/2362] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:35:17.462612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436c3d4-c8a4-4a7e-b8ff-36566c59065c None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:17.471805 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436c3d4-c8a4-4a7e-b8ff-36566c59065c None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:17.476548 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac29407-0e0d-4583-979b-d5bb59060e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:17.477185 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac29407-0e0d-4583-979b-d5bb59060e6f admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:17.477463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac29407-0e0d-4583-979b-d5bb59060e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:17.477735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac29407-0e0d-4583-979b-d5bb59060e6f admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:17.478412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac29407-0e0d-4583-979b-d5bb59060e6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:17.480309 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac29407-0e0d-4583-979b-d5bb59060e6f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:17.504988 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1202/2363] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:35:17 2022] POST /identity/v3/projects => generated 385 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:35:17.509049 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56659a53-2f48-4d59-bba2-5146f307878f admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:17.519356 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56659a53-2f48-4d59-bba2-5146f307878f admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:17.572351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b929f-dcfe-4fe8-8ef5-f2c976acf652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:17.572944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b929f-dcfe-4fe8-8ef5-f2c976acf652 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:17.573229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b929f-dcfe-4fe8-8ef5-f2c976acf652 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:17.573501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b929f-dcfe-4fe8-8ef5-f2c976acf652 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:17.574181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b929f-dcfe-4fe8-8ef5-f2c976acf652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:17.576123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b929f-dcfe-4fe8-8ef5-f2c976acf652 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:17.597781 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1162/2364] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:35:17 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:35:17.601806 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac29407-0e0d-4583-979b-d5bb59060e6f admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:17.612030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac29407-0e0d-4583-979b-d5bb59060e6f admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:17.616612 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5ac6-226b-4b75-b776-41e3c01a24e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:17.617262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5ac6-226b-4b75-b776-41e3c01a24e7 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:17.617543 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5ac6-226b-4b75-b776-41e3c01a24e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:17.617814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5ac6-226b-4b75-b776-41e3c01a24e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:17.618554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f5ac6-226b-4b75-b776-41e3c01a24e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:17.620224 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f5ac6-226b-4b75-b776-41e3c01a24e7 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:17.634967 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1203/2365] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:35:17.641405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b929f-dcfe-4fe8-8ef5-f2c976acf652 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:17.656252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b929f-dcfe-4fe8-8ef5-f2c976acf652 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:17.660987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e5c05-014e-443f-878a-e6d24706743e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:17.661646 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e5c05-014e-443f-878a-e6d24706743e admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:17.661932 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e5c05-014e-443f-878a-e6d24706743e admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:17.662230 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e5c05-014e-443f-878a-e6d24706743e admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:17.662958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e5c05-014e-443f-878a-e6d24706743e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:17.664577 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e5c05-014e-443f-878a-e6d24706743e admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:17.674944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1163/2366] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:35:17.679048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5ac6-226b-4b75-b776-41e3c01a24e7 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:17.689384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5ac6-226b-4b75-b776-41e3c01a24e7 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:17.694402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf0eaf-70b2-43f1-9b92-95d65964c09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:17.695329 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf0eaf-70b2-43f1-9b92-95d65964c09b admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:17.695600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf0eaf-70b2-43f1-9b92-95d65964c09b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:17.695872 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf0eaf-70b2-43f1-9b92-95d65964c09b admin admin] PATH_INFO: `/v3/projects/f1906e42c698453d9d0f95f6e74a3b55/users/08882cdf6f6a445f921198b818dc21a9/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:17.710810 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf0eaf-70b2-43f1-9b92-95d65964c09b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1906e42c698453d9d0f95f6e74a3b55, user_id=08882cdf6f6a445f921198b818dc21a9, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:17.713469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf0eaf-70b2-43f1-9b92-95d65964c09b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:17.725636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1204/2367] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:35:17 2022] PUT /identity/v3/projects/f1906e42c698453d9d0f95f6e74a3b55/users/08882cdf6f6a445f921198b818dc21a9/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:17.737123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a7732-4e7e-4800-a620-09cec41ddfa6 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:17.737411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a7732-4e7e-4800-a620-09cec41ddfa6 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:17.737684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a7732-4e7e-4800-a620-09cec41ddfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:17.764125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-bf2a7732-4e7e-4800-a620-09cec41ddfa6 None None] MFA Rules not processed for user `08882cdf6f6a445f921198b818dc21a9`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:17.765291 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-bf2a7732-4e7e-4800-a620-09cec41ddfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:17.826512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1164/2368] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:17.837294 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf0eaf-70b2-43f1-9b92-95d65964c09b admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:17.838939 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2ddf0eaf-70b2-43f1-9b92-95d65964c09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:17.894057 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf0eaf-70b2-43f1-9b92-95d65964c09b admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:17.897643 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6608c-b13b-405d-8aa3-46e2ba2726bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:17.898273 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6608c-b13b-405d-8aa3-46e2ba2726bd service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:17.898550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6608c-b13b-405d-8aa3-46e2ba2726bd service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:17.898822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6608c-b13b-405d-8aa3-46e2ba2726bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:17.906825 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e6608c-b13b-405d-8aa3-46e2ba2726bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:17.908244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e6608c-b13b-405d-8aa3-46e2ba2726bd service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:17.916657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1205/2369] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:35:25.251419 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a7732-4e7e-4800-a620-09cec41ddfa6 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:25.312611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a7732-4e7e-4800-a620-09cec41ddfa6 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:25.317216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f66fa-4824-4e36-9cf3-c4f1109be9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:25.318098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f66fa-4824-4e36-9cf3-c4f1109be9d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:25.318376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f66fa-4824-4e36-9cf3-c4f1109be9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:25.318646 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f66fa-4824-4e36-9cf3-c4f1109be9d8 admin admin] PATH_INFO: `/v3/users/08882cdf6f6a445f921198b818dc21a9` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:25.320005 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f66fa-4824-4e36-9cf3-c4f1109be9d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08882cdf6f6a445f921198b818dc21a9)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:25.321901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f66fa-4824-4e36-9cf3-c4f1109be9d8 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:25.370380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-0c2f66fa-4824-4e36-9cf3-c4f1109be9d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08882cdf6f6a445f921198b818dc21a9'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:25.377221 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-0c2f66fa-4824-4e36-9cf3-c4f1109be9d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08882cdf6f6a445f921198b818dc21a9'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:25.377904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-0c2f66fa-4824-4e36-9cf3-c4f1109be9d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08882cdf6f6a445f921198b818dc21a9'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:25.384881 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-0c2f66fa-4824-4e36-9cf3-c4f1109be9d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08882cdf6f6a445f921198b818dc21a9'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:25.385677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-0c2f66fa-4824-4e36-9cf3-c4f1109be9d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08882cdf6f6a445f921198b818dc21a9'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:25.402787 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1165/2370] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:35:25 2022] DELETE /identity/v3/users/08882cdf6f6a445f921198b818dc21a9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:25.407350 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f71662-499f-4704-8e05-8e99f23efd1b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:25.407839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f71662-499f-4704-8e05-8e99f23efd1b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:25.407941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f71662-499f-4704-8e05-8e99f23efd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:25.424744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b0f71662-499f-4704-8e05-8e99f23efd1b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:25.425910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b0f71662-499f-4704-8e05-8e99f23efd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:25.481462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1206/2371] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:35:25.490969 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f66fa-4824-4e36-9cf3-c4f1109be9d8 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:25.492384 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-0c2f66fa-4824-4e36-9cf3-c4f1109be9d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:25.547693 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f66fa-4824-4e36-9cf3-c4f1109be9d8 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:25.551331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42f75e-e07e-44d9-952b-fd1dd0050492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:25.551966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42f75e-e07e-44d9-952b-fd1dd0050492 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:25.552250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42f75e-e07e-44d9-952b-fd1dd0050492 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:25.552529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42f75e-e07e-44d9-952b-fd1dd0050492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:25.561313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42f75e-e07e-44d9-952b-fd1dd0050492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:25.562714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42f75e-e07e-44d9-952b-fd1dd0050492 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:25.571960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1166/2372] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:35:25.836459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f71662-499f-4704-8e05-8e99f23efd1b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:25.838006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b0f71662-499f-4704-8e05-8e99f23efd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:25.850001 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f71662-499f-4704-8e05-8e99f23efd1b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:25.854700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd05f9-df6a-4770-bb74-705a3338ab95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:25.855615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd05f9-df6a-4770-bb74-705a3338ab95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:25.855899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd05f9-df6a-4770-bb74-705a3338ab95 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:25.856174 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd05f9-df6a-4770-bb74-705a3338ab95 admin admin] PATH_INFO: `/v3/projects/f1906e42c698453d9d0f95f6e74a3b55` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:25.857374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd05f9-df6a-4770-bb74-705a3338ab95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1906e42c698453d9d0f95f6e74a3b55)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:25.859267 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd05f9-df6a-4770-bb74-705a3338ab95 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:25.904845 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-63fd05f9-df6a-4770-bb74-705a3338ab95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1906e42c698453d9d0f95f6e74a3b55', 'request_id': 'req-63fd05f9-df6a-4770-bb74-705a3338ab95'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:25.910079 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-63fd05f9-df6a-4770-bb74-705a3338ab95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1906e42c698453d9d0f95f6e74a3b55', 'request_id': 'req-63fd05f9-df6a-4770-bb74-705a3338ab95'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:25.917834 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-63fd05f9-df6a-4770-bb74-705a3338ab95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1906e42c698453d9d0f95f6e74a3b55', 'request_id': 'req-63fd05f9-df6a-4770-bb74-705a3338ab95'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:25.928228 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-63fd05f9-df6a-4770-bb74-705a3338ab95 admin admin] The token cache is being invalidate because project f1906e42c698453d9d0f95f6e74a3b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:35:25.928514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-63fd05f9-df6a-4770-bb74-705a3338ab95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:25.929138 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-63fd05f9-df6a-4770-bb74-705a3338ab95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:25.930650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1207/2373] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:35:25 2022] DELETE /identity/v3/projects/f1906e42c698453d9d0f95f6e74a3b55 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:25.935580 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4acd5-cec2-4416-bbce-3f4e8cf373cb None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:25.935859 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4acd5-cec2-4416-bbce-3f4e8cf373cb None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:25.936161 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4acd5-cec2-4416-bbce-3f4e8cf373cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:25.953842 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-40a4acd5-cec2-4416-bbce-3f4e8cf373cb None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:25.955014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-40a4acd5-cec2-4416-bbce-3f4e8cf373cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:26.019227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1167/2374] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:26.026901 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a14fc5-8a59-465b-b393-517fa5e85bf9 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:26.027242 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a14fc5-8a59-465b-b393-517fa5e85bf9 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:26.027525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a14fc5-8a59-465b-b393-517fa5e85bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:26.045088 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-a1a14fc5-8a59-465b-b393-517fa5e85bf9 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:26.046267 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a1a14fc5-8a59-465b-b393-517fa5e85bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:26.054700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1208/2375] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:26.058707 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4acd5-cec2-4416-bbce-3f4e8cf373cb None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:26.068646 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4acd5-cec2-4416-bbce-3f4e8cf373cb None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:26.073149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7235be-ea47-4cfd-822b-07fe71821a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:26.073790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7235be-ea47-4cfd-822b-07fe71821a6f admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:26.074069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7235be-ea47-4cfd-822b-07fe71821a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:26.074339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7235be-ea47-4cfd-822b-07fe71821a6f admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:26.075049 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7235be-ea47-4cfd-822b-07fe71821a6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:35:26.075384 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7235be-ea47-4cfd-822b-07fe71821a6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:26.076942 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7235be-ea47-4cfd-822b-07fe71821a6f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:26.087834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1168/2376] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:35:26.091786 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a14fc5-8a59-465b-b393-517fa5e85bf9 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:26.100766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a14fc5-8a59-465b-b393-517fa5e85bf9 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:26.105376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f5916-9866-42c4-ad76-0cb02db9e99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:26.106015 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f5916-9866-42c4-ad76-0cb02db9e99b admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:26.106300 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f5916-9866-42c4-ad76-0cb02db9e99b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:26.106578 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f5916-9866-42c4-ad76-0cb02db9e99b admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:26.107299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f5916-9866-42c4-ad76-0cb02db9e99b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:26.109175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f5916-9866-42c4-ad76-0cb02db9e99b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:26.134753 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1209/2377] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:35:26 2022] POST /identity/v3/projects => generated 371 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:35:26.138805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7235be-ea47-4cfd-822b-07fe71821a6f admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:26.148322 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7235be-ea47-4cfd-822b-07fe71821a6f admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:26.202209 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d39bd-8c12-4ee8-b7ef-0d257ef0e37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:26.202875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d39bd-8c12-4ee8-b7ef-0d257ef0e37c admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:26.203160 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d39bd-8c12-4ee8-b7ef-0d257ef0e37c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:26.203471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d39bd-8c12-4ee8-b7ef-0d257ef0e37c admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:26.204143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d39bd-8c12-4ee8-b7ef-0d257ef0e37c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:26.206030 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d39bd-8c12-4ee8-b7ef-0d257ef0e37c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:26.224958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1169/2378] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:35:26 2022] POST /identity/v3/users => generated 286 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:35:26.229077 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f5916-9866-42c4-ad76-0cb02db9e99b admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:26.239602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f5916-9866-42c4-ad76-0cb02db9e99b admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:26.244221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3af99-3f9e-4d8d-8a5e-e20bb30b8add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:26.244873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3af99-3f9e-4d8d-8a5e-e20bb30b8add admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:26.245166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3af99-3f9e-4d8d-8a5e-e20bb30b8add admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:26.245432 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3af99-3f9e-4d8d-8a5e-e20bb30b8add admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:26.246177 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3af99-3f9e-4d8d-8a5e-e20bb30b8add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:26.248058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3af99-3f9e-4d8d-8a5e-e20bb30b8add admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:26.258770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1210/2379] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:35:26.262717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d39bd-8c12-4ee8-b7ef-0d257ef0e37c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:26.272416 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d39bd-8c12-4ee8-b7ef-0d257ef0e37c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:26.277119 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caba50-d952-42b8-9a99-95a145cbb122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:26.277792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caba50-d952-42b8-9a99-95a145cbb122 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:26.278071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caba50-d952-42b8-9a99-95a145cbb122 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:26.278343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caba50-d952-42b8-9a99-95a145cbb122 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:26.279057 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caba50-d952-42b8-9a99-95a145cbb122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:26.280659 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caba50-d952-42b8-9a99-95a145cbb122 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:26.292808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1170/2380] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:35:26.296856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3af99-3f9e-4d8d-8a5e-e20bb30b8add admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:26.306969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3af99-3f9e-4d8d-8a5e-e20bb30b8add admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:26.311524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513e0d9-ce63-42ca-a748-11d32ddf1a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:26.312403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513e0d9-ce63-42ca-a748-11d32ddf1a0e admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:26.312666 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513e0d9-ce63-42ca-a748-11d32ddf1a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:26.312936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513e0d9-ce63-42ca-a748-11d32ddf1a0e admin admin] PATH_INFO: `/v3/projects/83477e7221724bae9e0a32ea2447f181/users/70c5e12cec3a4446a8152caf028ed75b/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:26.327580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513e0d9-ce63-42ca-a748-11d32ddf1a0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=83477e7221724bae9e0a32ea2447f181, user_id=70c5e12cec3a4446a8152caf028ed75b, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:26.330209 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513e0d9-ce63-42ca-a748-11d32ddf1a0e admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:26.342624 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1211/2381] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:35:26 2022] PUT /identity/v3/projects/83477e7221724bae9e0a32ea2447f181/users/70c5e12cec3a4446a8152caf028ed75b/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:26.352616 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4157a-2dea-4999-b679-9cff39f7b80c None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:26.352975 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4157a-2dea-4999-b679-9cff39f7b80c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:26.353320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4157a-2dea-4999-b679-9cff39f7b80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:26.380872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-22f4157a-2dea-4999-b679-9cff39f7b80c None None] MFA Rules not processed for user `70c5e12cec3a4446a8152caf028ed75b`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:26.382079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-22f4157a-2dea-4999-b679-9cff39f7b80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:26.444758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1171/2382] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:26.456120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513e0d9-ce63-42ca-a748-11d32ddf1a0e admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:26.457859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-0513e0d9-ce63-42ca-a748-11d32ddf1a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:26.516770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513e0d9-ce63-42ca-a748-11d32ddf1a0e admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:26.520346 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1cb2f-8ac6-4af3-8061-4d6d39591898 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cf284e81ba4042b66c0bc189f1ae2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:26.520983 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1cb2f-8ac6-4af3-8061-4d6d39591898 service glance] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:26.521262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1cb2f-8ac6-4af3-8061-4d6d39591898 service glance] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:26.521535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1cb2f-8ac6-4af3-8061-4d6d39591898 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:26.529460 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1cb2f-8ac6-4af3-8061-4d6d39591898 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:26.530792 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1cb2f-8ac6-4af3-8061-4d6d39591898 service glance] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:26.539760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1212/2383] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:35:31.294936 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4157a-2dea-4999-b679-9cff39f7b80c None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:31.296553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-22f4157a-2dea-4999-b679-9cff39f7b80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:31.372236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4157a-2dea-4999-b679-9cff39f7b80c None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:31.378974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8faee87-8f8a-4b5c-a4c4-aefbc0243bc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:31.379782 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faee87-8f8a-4b5c-a4c4-aefbc0243bc3 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:31.380177 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faee87-8f8a-4b5c-a4c4-aefbc0243bc3 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:31.380452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faee87-8f8a-4b5c-a4c4-aefbc0243bc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:31.382264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c8faee87-8f8a-4b5c-a4c4-aefbc0243bc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:31.394606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8faee87-8f8a-4b5c-a4c4-aefbc0243bc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:31.396202 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8faee87-8f8a-4b5c-a4c4-aefbc0243bc3 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:31.406389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1172/2384] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:35:32.650168 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbce315-85c8-4c91-88a4-1abc9eb0976f None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:32.651050 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbce315-85c8-4c91-88a4-1abc9eb0976f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:32.651050 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbce315-85c8-4c91-88a4-1abc9eb0976f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:32.668046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2dbce315-85c8-4c91-88a4-1abc9eb0976f None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:32.669219 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2dbce315-85c8-4c91-88a4-1abc9eb0976f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:32.721861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1213/2385] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:35:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:32.769713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8faee87-8f8a-4b5c-a4c4-aefbc0243bc3 service nova] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:32.771120 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c8faee87-8f8a-4b5c-a4c4-aefbc0243bc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:32.838040 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8faee87-8f8a-4b5c-a4c4-aefbc0243bc3 service nova] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:32.843732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a61fc0-cb7c-4354-a3a0-eef9d997b694 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:32.844399 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a61fc0-cb7c-4354-a3a0-eef9d997b694 service swift] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:32.844969 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a61fc0-cb7c-4354-a3a0-eef9d997b694 service swift] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:32.845415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a61fc0-cb7c-4354-a3a0-eef9d997b694 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:32.856294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a61fc0-cb7c-4354-a3a0-eef9d997b694 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:32.858164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a61fc0-cb7c-4354-a3a0-eef9d997b694 service swift] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:32.866121 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1173/2386] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:35:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 18:35:50.428583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e284fce-0777-42e9-84cf-c86203fc23d6 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:50.429043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e284fce-0777-42e9-84cf-c86203fc23d6 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:50.429239 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e284fce-0777-42e9-84cf-c86203fc23d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:50.447061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9e284fce-0777-42e9-84cf-c86203fc23d6 None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:50.448243 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9e284fce-0777-42e9-84cf-c86203fc23d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:50.455347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1214/2387] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:35:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:50.467698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a61fc0-cb7c-4354-a3a0-eef9d997b694 service swift] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:50.477850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a61fc0-cb7c-4354-a3a0-eef9d997b694 service swift] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:50.482831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7906230-18e7-4732-8273-60d0048d533e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:50.483492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7906230-18e7-4732-8273-60d0048d533e service swift] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:50.483776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7906230-18e7-4732-8273-60d0048d533e service swift] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:50.484054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7906230-18e7-4732-8273-60d0048d533e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:50.492264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7906230-18e7-4732-8273-60d0048d533e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:50.493811 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7906230-18e7-4732-8273-60d0048d533e service swift] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:50.502252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1174/2388] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:35:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 18:35:53.351747 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e284fce-0777-42e9-84cf-c86203fc23d6 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:53.430602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e284fce-0777-42e9-84cf-c86203fc23d6 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:53.435387 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44079d4f-eda9-47db-ba6f-0717651a8eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:53.436288 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44079d4f-eda9-47db-ba6f-0717651a8eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:53.436567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44079d4f-eda9-47db-ba6f-0717651a8eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:53.436840 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44079d4f-eda9-47db-ba6f-0717651a8eb6 admin admin] PATH_INFO: `/v3/users/70c5e12cec3a4446a8152caf028ed75b` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:53.438058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44079d4f-eda9-47db-ba6f-0717651a8eb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70c5e12cec3a4446a8152caf028ed75b)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:53.440042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44079d4f-eda9-47db-ba6f-0717651a8eb6 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:53.501974 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-44079d4f-eda9-47db-ba6f-0717651a8eb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70c5e12cec3a4446a8152caf028ed75b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:53.502776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-44079d4f-eda9-47db-ba6f-0717651a8eb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70c5e12cec3a4446a8152caf028ed75b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:53.512342 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-44079d4f-eda9-47db-ba6f-0717651a8eb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70c5e12cec3a4446a8152caf028ed75b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:53.520532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-44079d4f-eda9-47db-ba6f-0717651a8eb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70c5e12cec3a4446a8152caf028ed75b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:53.521493 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-44079d4f-eda9-47db-ba6f-0717651a8eb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70c5e12cec3a4446a8152caf028ed75b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:53.539728 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1215/2389] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:35:53 2022] DELETE /identity/v3/users/70c5e12cec3a4446a8152caf028ed75b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:53.544709 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb22873-aba9-4350-a674-8e43b20ab88a None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:53.545252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb22873-aba9-4350-a674-8e43b20ab88a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:53.545252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb22873-aba9-4350-a674-8e43b20ab88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:53.562856 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-1fb22873-aba9-4350-a674-8e43b20ab88a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:53.564092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1fb22873-aba9-4350-a674-8e43b20ab88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:53.630965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1175/2390] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:53.641003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44079d4f-eda9-47db-ba6f-0717651a8eb6 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:53.642397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-44079d4f-eda9-47db-ba6f-0717651a8eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:53.712071 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44079d4f-eda9-47db-ba6f-0717651a8eb6 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:53.715859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c6967-2b50-4a1e-b979-4a9accac0bcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:53.716595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c6967-2b50-4a1e-b979-4a9accac0bcd service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:53.716941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c6967-2b50-4a1e-b979-4a9accac0bcd service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:53.717220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c6967-2b50-4a1e-b979-4a9accac0bcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:53.729548 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c6967-2b50-4a1e-b979-4a9accac0bcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:53.730934 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c6967-2b50-4a1e-b979-4a9accac0bcd service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:53.741644 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1216/2391] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:35:54.038343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb22873-aba9-4350-a674-8e43b20ab88a None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:54.040383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1fb22873-aba9-4350-a674-8e43b20ab88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:54.056885 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb22873-aba9-4350-a674-8e43b20ab88a None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:54.061575 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:54.062479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:54.062802 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:54.063165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df admin admin] PATH_INFO: `/v3/projects/83477e7221724bae9e0a32ea2447f181` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:54.064502 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df admin admin] RBAC: Authorizing `identity:delete_project(project_id=83477e7221724bae9e0a32ea2447f181)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:54.068541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:54.130475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83477e7221724bae9e0a32ea2447f181', 'request_id': 'req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:54.135835 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83477e7221724bae9e0a32ea2447f181', 'request_id': 'req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:54.148530 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83477e7221724bae9e0a32ea2447f181', 'request_id': 'req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:54.163886 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df admin admin] The token cache is being invalidate because project 83477e7221724bae9e0a32ea2447f181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:35:54.164185 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:54.164833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ec5c23a4-671d-4b4c-b63a-d6956d93a1df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:35:54.166597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1176/2392] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:35:54 2022] DELETE /identity/v3/projects/83477e7221724bae9e0a32ea2447f181 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:35:54.171261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d192276-73ee-45cf-9801-0945bcfdeff8 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:54.171617 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d192276-73ee-45cf-9801-0945bcfdeff8 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:54.171913 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d192276-73ee-45cf-9801-0945bcfdeff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:54.204483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-1d192276-73ee-45cf-9801-0945bcfdeff8 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:54.205687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-1d192276-73ee-45cf-9801-0945bcfdeff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:54.273985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1217/2393] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:54.282130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6e807-3c5d-4f05-856d-9f5cca512297 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:54.282130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6e807-3c5d-4f05-856d-9f5cca512297 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:54.282593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6e807-3c5d-4f05-856d-9f5cca512297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:54.313332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-e2d6e807-3c5d-4f05-856d-9f5cca512297 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:54.314541 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e2d6e807-3c5d-4f05-856d-9f5cca512297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:54.323810 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1177/2394] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:54.327975 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d192276-73ee-45cf-9801-0945bcfdeff8 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:54.337926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d192276-73ee-45cf-9801-0945bcfdeff8 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:54.342556 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235abc43-b588-4f21-b110-7eb8dd44ffa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:54.343358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235abc43-b588-4f21-b110-7eb8dd44ffa5 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:54.343703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235abc43-b588-4f21-b110-7eb8dd44ffa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:54.344048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235abc43-b588-4f21-b110-7eb8dd44ffa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:54.344831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235abc43-b588-4f21-b110-7eb8dd44ffa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:35:54.345201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235abc43-b588-4f21-b110-7eb8dd44ffa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:54.346838 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235abc43-b588-4f21-b110-7eb8dd44ffa5 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:54.362104 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1218/2395] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:35:54.364587 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6e807-3c5d-4f05-856d-9f5cca512297 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:54.374511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6e807-3c5d-4f05-856d-9f5cca512297 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:54.379465 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88bbab-1336-4824-bf5d-ad0748da5922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:54.380134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88bbab-1336-4824-bf5d-ad0748da5922 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:54.380794 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88bbab-1336-4824-bf5d-ad0748da5922 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:54.381653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88bbab-1336-4824-bf5d-ad0748da5922 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:54.383219 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88bbab-1336-4824-bf5d-ad0748da5922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:54.387205 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88bbab-1336-4824-bf5d-ad0748da5922 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:54.425550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1178/2396] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:35:54 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:35:54.430272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235abc43-b588-4f21-b110-7eb8dd44ffa5 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:54.441978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235abc43-b588-4f21-b110-7eb8dd44ffa5 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:54.502704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a014b-6e1a-4dbb-b52b-6467e6d7840e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:54.503806 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a014b-6e1a-4dbb-b52b-6467e6d7840e admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:54.504186 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a014b-6e1a-4dbb-b52b-6467e6d7840e admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:54.504558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a014b-6e1a-4dbb-b52b-6467e6d7840e admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:54.505345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a014b-6e1a-4dbb-b52b-6467e6d7840e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:54.507378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a014b-6e1a-4dbb-b52b-6467e6d7840e admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:54.510995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88bbab-1336-4824-bf5d-ad0748da5922 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:54.512709 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-db88bbab-1336-4824-bf5d-ad0748da5922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:54.532859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1219/2397] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:35:54 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:35:54.537181 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a014b-6e1a-4dbb-b52b-6467e6d7840e admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:54.547567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a014b-6e1a-4dbb-b52b-6467e6d7840e admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:54.552366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee84fa-df69-4282-b4b8-150ca79cb42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:54.553108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee84fa-df69-4282-b4b8-150ca79cb42a admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:54.553773 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee84fa-df69-4282-b4b8-150ca79cb42a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:54.554235 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee84fa-df69-4282-b4b8-150ca79cb42a admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:54.555965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee84fa-df69-4282-b4b8-150ca79cb42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:54.560672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee84fa-df69-4282-b4b8-150ca79cb42a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:54.575360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1220/2398] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:35:54.577654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee84fa-df69-4282-b4b8-150ca79cb42a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:54.596284 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee84fa-df69-4282-b4b8-150ca79cb42a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:54.602684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa8fb5-b58e-41b9-a881-51ead6cc29ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:54.603631 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa8fb5-b58e-41b9-a881-51ead6cc29ef admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:54.604078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa8fb5-b58e-41b9-a881-51ead6cc29ef admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:54.604634 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa8fb5-b58e-41b9-a881-51ead6cc29ef admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:54.605499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa8fb5-b58e-41b9-a881-51ead6cc29ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:54.605827 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88bbab-1336-4824-bf5d-ad0748da5922 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:54.607773 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa8fb5-b58e-41b9-a881-51ead6cc29ef admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:54.611752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23958f0-d1ce-4821-8ed5-29ac1de2bc72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:54.612610 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23958f0-d1ce-4821-8ed5-29ac1de2bc72 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:54.613224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23958f0-d1ce-4821-8ed5-29ac1de2bc72 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:54.613512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23958f0-d1ce-4821-8ed5-29ac1de2bc72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:54.615427 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e23958f0-d1ce-4821-8ed5-29ac1de2bc72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:54.620047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1221/2399] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:35:54.625484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa8fb5-b58e-41b9-a881-51ead6cc29ef admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:54.640893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa8fb5-b58e-41b9-a881-51ead6cc29ef admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:54.647442 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ef35c-bfe6-43a0-812b-8ff5a8530f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:54.648580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ef35c-bfe6-43a0-812b-8ff5a8530f2c admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:54.648960 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ef35c-bfe6-43a0-812b-8ff5a8530f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:54.649500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ef35c-bfe6-43a0-812b-8ff5a8530f2c admin admin] PATH_INFO: `/v3/projects/64dd788fefab4eeab253b0f0d341fd2d/users/b2e8f8cb607649c0836a9c709791fc0e/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:54.667137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ef35c-bfe6-43a0-812b-8ff5a8530f2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=64dd788fefab4eeab253b0f0d341fd2d, user_id=b2e8f8cb607649c0836a9c709791fc0e, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:54.672227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ef35c-bfe6-43a0-812b-8ff5a8530f2c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:54.695936 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1222/2400] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:35:54 2022] PUT /identity/v3/projects/64dd788fefab4eeab253b0f0d341fd2d/users/b2e8f8cb607649c0836a9c709791fc0e/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:35:54.696185 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23958f0-d1ce-4821-8ed5-29ac1de2bc72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:54.696185 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23958f0-d1ce-4821-8ed5-29ac1de2bc72 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:54.698168 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1ba95-ed71-4cc2-a3d8-66211b4aca4d None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:54.698686 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1ba95-ed71-4cc2-a3d8-66211b4aca4d None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:54.699029 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1ba95-ed71-4cc2-a3d8-66211b4aca4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:54.719405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1179/2401] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:35:54.719720 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-edd1ba95-ed71-4cc2-a3d8-66211b4aca4d None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:54.721030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-edd1ba95-ed71-4cc2-a3d8-66211b4aca4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:54.792740 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1223/2402] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:35:54.803391 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23958f0-d1ce-4821-8ed5-29ac1de2bc72 service nova] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:54.804851 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e23958f0-d1ce-4821-8ed5-29ac1de2bc72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:54.894817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23958f0-d1ce-4821-8ed5-29ac1de2bc72 service nova] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:54.948594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01928b1e-7029-4328-b3ba-15f4dc760d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:54.949421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01928b1e-7029-4328-b3ba-15f4dc760d5b service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:54.949796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01928b1e-7029-4328-b3ba-15f4dc760d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:54.950163 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01928b1e-7029-4328-b3ba-15f4dc760d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:54.960318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01928b1e-7029-4328-b3ba-15f4dc760d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:54.961913 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01928b1e-7029-4328-b3ba-15f4dc760d5b service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:54.976601 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1180/2403] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:35:55.729100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce9aa3-d185-4e51-8e60-a467e9f2ef4e None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:55.729645 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce9aa3-d185-4e51-8e60-a467e9f2ef4e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:55.729951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce9aa3-d185-4e51-8e60-a467e9f2ef4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:55.766712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-12ce9aa3-d185-4e51-8e60-a467e9f2ef4e None None] MFA Rules not processed for user `b2e8f8cb607649c0836a9c709791fc0e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:55.768083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-12ce9aa3-d185-4e51-8e60-a467e9f2ef4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:55.881126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1224/2404] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:35:55.890664 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01928b1e-7029-4328-b3ba-15f4dc760d5b service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:35:55.892141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-01928b1e-7029-4328-b3ba-15f4dc760d5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:55.956005 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01928b1e-7029-4328-b3ba-15f4dc760d5b service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:35:55.970346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451571bb-3b0c-4fb3-8dd3-c57250cfd9e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:35:55.971154 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451571bb-3b0c-4fb3-8dd3-c57250cfd9e6 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:55.971533 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451571bb-3b0c-4fb3-8dd3-c57250cfd9e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:55.971882 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451571bb-3b0c-4fb3-8dd3-c57250cfd9e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:55.996787 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451571bb-3b0c-4fb3-8dd3-c57250cfd9e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:35:55.998356 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451571bb-3b0c-4fb3-8dd3-c57250cfd9e6 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:35:56.009890 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1181/2405] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:35:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:35:57.419485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66015ab-eb1e-4006-8785-7e2eb2c4a5b3 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:35:57.420084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66015ab-eb1e-4006-8785-7e2eb2c4a5b3 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:35:57.420084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66015ab-eb1e-4006-8785-7e2eb2c4a5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:35:57.439670 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-e66015ab-eb1e-4006-8785-7e2eb2c4a5b3 None None] MFA Rules not processed for user `b2e8f8cb607649c0836a9c709791fc0e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:35:57.440881 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e66015ab-eb1e-4006-8785-7e2eb2c4a5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:35:57.450224 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1225/2406] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:35:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:02.478749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451571bb-3b0c-4fb3-8dd3-c57250cfd9e6 service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:02.480726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-451571bb-3b0c-4fb3-8dd3-c57250cfd9e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:02.565098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451571bb-3b0c-4fb3-8dd3-c57250cfd9e6 service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:02.574192 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62892e-f528-46ec-86da-0348d2ea0ac7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:02.574854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62892e-f528-46ec-86da-0348d2ea0ac7 service nova] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:02.575139 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62892e-f528-46ec-86da-0348d2ea0ac7 service nova] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:02.575484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62892e-f528-46ec-86da-0348d2ea0ac7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:02.578087 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ab62892e-f528-46ec-86da-0348d2ea0ac7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:02.594251 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62892e-f528-46ec-86da-0348d2ea0ac7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:02.595922 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62892e-f528-46ec-86da-0348d2ea0ac7 service nova] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:02.607271 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1182/2407] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:07.100250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1630caf-3c78-4368-9328-dd10455279cf None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:07.102758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1630caf-3c78-4368-9328-dd10455279cf None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:07.102758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1630caf-3c78-4368-9328-dd10455279cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:07.124374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-c1630caf-3c78-4368-9328-dd10455279cf None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:07.125843 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c1630caf-3c78-4368-9328-dd10455279cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:07.187061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1226/2408] 10.222.0.25 () {58 vars in 967 bytes} [Mon May 9 18:36:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:07.200322 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62892e-f528-46ec-86da-0348d2ea0ac7 service nova] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:07.201720 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ab62892e-f528-46ec-86da-0348d2ea0ac7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:07.275410 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62892e-f528-46ec-86da-0348d2ea0ac7 service nova] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:07.282170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460da7d-4cdf-482e-ac0c-691f0539d7ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:07.282969 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460da7d-4cdf-482e-ac0c-691f0539d7ed service swift] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:07.283292 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460da7d-4cdf-482e-ac0c-691f0539d7ed service swift] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:07.283570 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460da7d-4cdf-482e-ac0c-691f0539d7ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:07.293037 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460da7d-4cdf-482e-ac0c-691f0539d7ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:07.294611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460da7d-4cdf-482e-ac0c-691f0539d7ed service swift] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:07.305611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1183/2409] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:36:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 09 18:36:12.727053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1630caf-3c78-4368-9328-dd10455279cf None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:12.737140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1630caf-3c78-4368-9328-dd10455279cf None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:12.740809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541d9c6-a703-4ac5-acc4-a2a09b0961cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:12.741468 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541d9c6-a703-4ac5-acc4-a2a09b0961cd service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:12.741748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541d9c6-a703-4ac5-acc4-a2a09b0961cd service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:12.742023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541d9c6-a703-4ac5-acc4-a2a09b0961cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:12.743793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-8541d9c6-a703-4ac5-acc4-a2a09b0961cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:12.756040 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541d9c6-a703-4ac5-acc4-a2a09b0961cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:12.757485 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541d9c6-a703-4ac5-acc4-a2a09b0961cd service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:12.767488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1227/2410] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:36:16.131906 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460da7d-4cdf-482e-ac0c-691f0539d7ed service swift] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:16.134283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f460da7d-4cdf-482e-ac0c-691f0539d7ed service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:16.151918 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460da7d-4cdf-482e-ac0c-691f0539d7ed service swift] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:16.157101 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9f957-3331-4882-8099-ec2831deaa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:16.157997 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9f957-3331-4882-8099-ec2831deaa1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:16.158278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9f957-3331-4882-8099-ec2831deaa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:16.158617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9f957-3331-4882-8099-ec2831deaa1b admin admin] PATH_INFO: `/v3/users/f99208187e4e4d2eb9f52107ae8f50f2` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:16.160669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9f957-3331-4882-8099-ec2831deaa1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f99208187e4e4d2eb9f52107ae8f50f2)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:16.162676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e9f957-3331-4882-8099-ec2831deaa1b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:16.214516 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-61e9f957-3331-4882-8099-ec2831deaa1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f99208187e4e4d2eb9f52107ae8f50f2'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:16.222116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-61e9f957-3331-4882-8099-ec2831deaa1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f99208187e4e4d2eb9f52107ae8f50f2'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:16.223624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-61e9f957-3331-4882-8099-ec2831deaa1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f99208187e4e4d2eb9f52107ae8f50f2'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:16.232126 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-61e9f957-3331-4882-8099-ec2831deaa1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f99208187e4e4d2eb9f52107ae8f50f2'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:16.233080 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-61e9f957-3331-4882-8099-ec2831deaa1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f99208187e4e4d2eb9f52107ae8f50f2'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:16.252482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1184/2411] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:16 2022] DELETE /identity/v3/users/f99208187e4e4d2eb9f52107ae8f50f2 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:16.439741 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541d9c6-a703-4ac5-acc4-a2a09b0961cd service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:16.450380 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541d9c6-a703-4ac5-acc4-a2a09b0961cd service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:16.512358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8eaf5e-2588-4382-886d-9286ad83a1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:16.513117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8eaf5e-2588-4382-886d-9286ad83a1e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:16.513395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8eaf5e-2588-4382-886d-9286ad83a1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:16.513691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8eaf5e-2588-4382-886d-9286ad83a1e0 admin admin] PATH_INFO: `/v3/projects/3582c1a023c645b38b91a5fa01be7667` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:16.514838 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8eaf5e-2588-4382-886d-9286ad83a1e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3582c1a023c645b38b91a5fa01be7667)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:16.516768 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8eaf5e-2588-4382-886d-9286ad83a1e0 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:16.561006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-0a8eaf5e-2588-4382-886d-9286ad83a1e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3582c1a023c645b38b91a5fa01be7667', 'request_id': 'req-0a8eaf5e-2588-4382-886d-9286ad83a1e0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:16.567048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-0a8eaf5e-2588-4382-886d-9286ad83a1e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3582c1a023c645b38b91a5fa01be7667', 'request_id': 'req-0a8eaf5e-2588-4382-886d-9286ad83a1e0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:16.574668 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-0a8eaf5e-2588-4382-886d-9286ad83a1e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3582c1a023c645b38b91a5fa01be7667', 'request_id': 'req-0a8eaf5e-2588-4382-886d-9286ad83a1e0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:16.588825 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-0a8eaf5e-2588-4382-886d-9286ad83a1e0 admin admin] The token cache is being invalidate because project 3582c1a023c645b38b91a5fa01be7667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:16.589115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-0a8eaf5e-2588-4382-886d-9286ad83a1e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:16.589786 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-0a8eaf5e-2588-4382-886d-9286ad83a1e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:16.591594 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1228/2412] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:16 2022] DELETE /identity/v3/projects/3582c1a023c645b38b91a5fa01be7667 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:16.596306 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d093027-8af0-4b6e-b346-61498196453b None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:16.596699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d093027-8af0-4b6e-b346-61498196453b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:16.597052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d093027-8af0-4b6e-b346-61498196453b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:16.620862 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-9d093027-8af0-4b6e-b346-61498196453b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:16.622353 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9d093027-8af0-4b6e-b346-61498196453b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:16.682568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1185/2413] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:16.691413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d469c-aced-4b41-8922-8fe9fe082f91 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:16.691909 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d469c-aced-4b41-8922-8fe9fe082f91 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:16.692189 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d469c-aced-4b41-8922-8fe9fe082f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:16.710122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-500d469c-aced-4b41-8922-8fe9fe082f91 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:16.711371 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-500d469c-aced-4b41-8922-8fe9fe082f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:16.719954 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1229/2414] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:16.724708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d093027-8af0-4b6e-b346-61498196453b None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:16.734255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d093027-8af0-4b6e-b346-61498196453b None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:16.739363 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052afec9-97cc-462b-afb5-c1e0caa5b344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:16.740017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052afec9-97cc-462b-afb5-c1e0caa5b344 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:16.740303 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052afec9-97cc-462b-afb5-c1e0caa5b344 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:16.740583 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052afec9-97cc-462b-afb5-c1e0caa5b344 admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:16.741304 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052afec9-97cc-462b-afb5-c1e0caa5b344 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:36:16.741613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052afec9-97cc-462b-afb5-c1e0caa5b344 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:16.743343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052afec9-97cc-462b-afb5-c1e0caa5b344 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:16.755951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1186/2415] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:36:16.760405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d469c-aced-4b41-8922-8fe9fe082f91 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:16.769588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d469c-aced-4b41-8922-8fe9fe082f91 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:16.774530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd87fe6-d9ef-4bbb-b11a-fe637d743f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:16.775316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd87fe6-d9ef-4bbb-b11a-fe637d743f99 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:16.775601 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd87fe6-d9ef-4bbb-b11a-fe637d743f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:16.775876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd87fe6-d9ef-4bbb-b11a-fe637d743f99 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:16.776983 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd87fe6-d9ef-4bbb-b11a-fe637d743f99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:16.778895 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd87fe6-d9ef-4bbb-b11a-fe637d743f99 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:16.805762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1230/2416] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:16 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:16.809824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052afec9-97cc-462b-afb5-c1e0caa5b344 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:16.820568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052afec9-97cc-462b-afb5-c1e0caa5b344 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:16.875602 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c3f20-43ce-4b83-96c3-7a1a2e291785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:16.876278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c3f20-43ce-4b83-96c3-7a1a2e291785 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:16.876562 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c3f20-43ce-4b83-96c3-7a1a2e291785 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:16.876840 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c3f20-43ce-4b83-96c3-7a1a2e291785 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:16.877529 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c3f20-43ce-4b83-96c3-7a1a2e291785 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:16.879954 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c3f20-43ce-4b83-96c3-7a1a2e291785 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:16.900134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1187/2417] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:16 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:16.904588 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd87fe6-d9ef-4bbb-b11a-fe637d743f99 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:16.914783 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd87fe6-d9ef-4bbb-b11a-fe637d743f99 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:16.919435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa20a1a-fd7a-40b6-9233-c29e5633083c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:16.920094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa20a1a-fd7a-40b6-9233-c29e5633083c admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:16.920376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa20a1a-fd7a-40b6-9233-c29e5633083c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:16.920650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa20a1a-fd7a-40b6-9233-c29e5633083c admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:16.921370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa20a1a-fd7a-40b6-9233-c29e5633083c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:16.922941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa20a1a-fd7a-40b6-9233-c29e5633083c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:16.933850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1231/2418] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:16.937987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c3f20-43ce-4b83-96c3-7a1a2e291785 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:16.948421 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c3f20-43ce-4b83-96c3-7a1a2e291785 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:16.952962 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c7d48-8116-482d-b2f7-51fe12f9f9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:16.953614 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7d48-8116-482d-b2f7-51fe12f9f9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:16.953893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7d48-8116-482d-b2f7-51fe12f9f9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:16.954165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7d48-8116-482d-b2f7-51fe12f9f9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:16.954913 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c7d48-8116-482d-b2f7-51fe12f9f9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:16.956521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c7d48-8116-482d-b2f7-51fe12f9f9f2 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:16.970109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1188/2419] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:16.971333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa20a1a-fd7a-40b6-9233-c29e5633083c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:16.981444 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa20a1a-fd7a-40b6-9233-c29e5633083c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:16.985987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e68f5c-11c9-4a85-93b0-1c093c86e634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:16.986851 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e68f5c-11c9-4a85-93b0-1c093c86e634 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:16.987126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e68f5c-11c9-4a85-93b0-1c093c86e634 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:16.987428 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e68f5c-11c9-4a85-93b0-1c093c86e634 admin admin] PATH_INFO: `/v3/projects/69d14b3d7e924f558cd44f894c41a70c/users/a55eedc9183c430c9db61fb07f168bec/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:17.003613 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e68f5c-11c9-4a85-93b0-1c093c86e634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d14b3d7e924f558cd44f894c41a70c, user_id=a55eedc9183c430c9db61fb07f168bec, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:17.006828 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e68f5c-11c9-4a85-93b0-1c093c86e634 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:17.019410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1232/2420] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:16 2022] PUT /identity/v3/projects/69d14b3d7e924f558cd44f894c41a70c/users/a55eedc9183c430c9db61fb07f168bec/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:17.029867 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf5124-7164-4ccf-9482-7cf2d571c542 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:17.030063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf5124-7164-4ccf-9482-7cf2d571c542 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:17.030338 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf5124-7164-4ccf-9482-7cf2d571c542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:17.057584 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-5fdf5124-7164-4ccf-9482-7cf2d571c542 None None] MFA Rules not processed for user `a55eedc9183c430c9db61fb07f168bec`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:17.058867 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-5fdf5124-7164-4ccf-9482-7cf2d571c542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:17.123401 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1189/2421] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:17.135006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e68f5c-11c9-4a85-93b0-1c093c86e634 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:17.136677 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-12e68f5c-11c9-4a85-93b0-1c093c86e634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:17.210346 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e68f5c-11c9-4a85-93b0-1c093c86e634 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:17.215986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44bc39-40e1-4583-a20e-76f878397b5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:17.216641 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44bc39-40e1-4583-a20e-76f878397b5b service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:17.216923 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44bc39-40e1-4583-a20e-76f878397b5b service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:17.217198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44bc39-40e1-4583-a20e-76f878397b5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:17.225479 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44bc39-40e1-4583-a20e-76f878397b5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:17.227043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44bc39-40e1-4583-a20e-76f878397b5b service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:17.236602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1233/2422] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:19.892060 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf5124-7164-4ccf-9482-7cf2d571c542 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:19.951968 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf5124-7164-4ccf-9482-7cf2d571c542 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:19.957213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de882b24-9206-46cb-abbe-a4b8bb906970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:19.958134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de882b24-9206-46cb-abbe-a4b8bb906970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:19.958426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de882b24-9206-46cb-abbe-a4b8bb906970 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:19.958704 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de882b24-9206-46cb-abbe-a4b8bb906970 admin admin] PATH_INFO: `/v3/users/a55eedc9183c430c9db61fb07f168bec` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:19.959947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de882b24-9206-46cb-abbe-a4b8bb906970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a55eedc9183c430c9db61fb07f168bec)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:19.961856 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de882b24-9206-46cb-abbe-a4b8bb906970 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:20.009506 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-de882b24-9206-46cb-abbe-a4b8bb906970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a55eedc9183c430c9db61fb07f168bec'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:20.016348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-de882b24-9206-46cb-abbe-a4b8bb906970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a55eedc9183c430c9db61fb07f168bec'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:20.017045 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-de882b24-9206-46cb-abbe-a4b8bb906970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a55eedc9183c430c9db61fb07f168bec'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:20.024443 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-de882b24-9206-46cb-abbe-a4b8bb906970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a55eedc9183c430c9db61fb07f168bec'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:20.025280 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-de882b24-9206-46cb-abbe-a4b8bb906970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a55eedc9183c430c9db61fb07f168bec'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:20.042395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1190/2423] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:19 2022] DELETE /identity/v3/users/a55eedc9183c430c9db61fb07f168bec => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:20.046817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5ac19-dd4a-4c41-a176-0a3a0e98d10f None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:20.047351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5ac19-dd4a-4c41-a176-0a3a0e98d10f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:20.047450 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5ac19-dd4a-4c41-a176-0a3a0e98d10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:20.066185 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-bcd5ac19-dd4a-4c41-a176-0a3a0e98d10f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:20.068552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-bcd5ac19-dd4a-4c41-a176-0a3a0e98d10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:20.130301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1234/2424] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:20.139970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de882b24-9206-46cb-abbe-a4b8bb906970 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:20.141346 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-de882b24-9206-46cb-abbe-a4b8bb906970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:20.199483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de882b24-9206-46cb-abbe-a4b8bb906970 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:20.203155 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f503e2b-2880-4e73-b6d0-c0fd3817988d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:20.203833 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f503e2b-2880-4e73-b6d0-c0fd3817988d service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:20.204114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f503e2b-2880-4e73-b6d0-c0fd3817988d service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:20.204389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f503e2b-2880-4e73-b6d0-c0fd3817988d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:20.213327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f503e2b-2880-4e73-b6d0-c0fd3817988d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:20.214736 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f503e2b-2880-4e73-b6d0-c0fd3817988d service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:20.224964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1191/2425] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:20.495896 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5ac19-dd4a-4c41-a176-0a3a0e98d10f None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:20.497671 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-bcd5ac19-dd4a-4c41-a176-0a3a0e98d10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:20.511988 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5ac19-dd4a-4c41-a176-0a3a0e98d10f None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:20.517014 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ce420-1a52-435b-b810-58e4e8dd4167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:20.517987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ce420-1a52-435b-b810-58e4e8dd4167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:20.518338 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ce420-1a52-435b-b810-58e4e8dd4167 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:20.518722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ce420-1a52-435b-b810-58e4e8dd4167 admin admin] PATH_INFO: `/v3/projects/69d14b3d7e924f558cd44f894c41a70c` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:20.520804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ce420-1a52-435b-b810-58e4e8dd4167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d14b3d7e924f558cd44f894c41a70c)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:20.523431 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ce420-1a52-435b-b810-58e4e8dd4167 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:20.570994 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-882ce420-1a52-435b-b810-58e4e8dd4167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d14b3d7e924f558cd44f894c41a70c', 'request_id': 'req-882ce420-1a52-435b-b810-58e4e8dd4167'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:20.576577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-882ce420-1a52-435b-b810-58e4e8dd4167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d14b3d7e924f558cd44f894c41a70c', 'request_id': 'req-882ce420-1a52-435b-b810-58e4e8dd4167'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:20.585230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-882ce420-1a52-435b-b810-58e4e8dd4167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d14b3d7e924f558cd44f894c41a70c', 'request_id': 'req-882ce420-1a52-435b-b810-58e4e8dd4167'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:20.596167 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-882ce420-1a52-435b-b810-58e4e8dd4167 admin admin] The token cache is being invalidate because project 69d14b3d7e924f558cd44f894c41a70c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:20.596470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-882ce420-1a52-435b-b810-58e4e8dd4167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:20.597087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-882ce420-1a52-435b-b810-58e4e8dd4167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:20.598654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1235/2426] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:20 2022] DELETE /identity/v3/projects/69d14b3d7e924f558cd44f894c41a70c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:20.603560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406f70f-f8b0-4416-8895-36bfee338e6d None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:20.603854 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406f70f-f8b0-4416-8895-36bfee338e6d None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:20.604132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406f70f-f8b0-4416-8895-36bfee338e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:20.622127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-2406f70f-f8b0-4416-8895-36bfee338e6d None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:20.623434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-2406f70f-f8b0-4416-8895-36bfee338e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:20.686366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1192/2427] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:20.694634 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973855b-eced-4d45-a50b-6b7170034e52 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:20.694916 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973855b-eced-4d45-a50b-6b7170034e52 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:20.695203 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973855b-eced-4d45-a50b-6b7170034e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:20.713704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-a973855b-eced-4d45-a50b-6b7170034e52 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:20.714890 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a973855b-eced-4d45-a50b-6b7170034e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:20.722736 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1236/2428] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:20.727016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406f70f-f8b0-4416-8895-36bfee338e6d None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:20.736594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406f70f-f8b0-4416-8895-36bfee338e6d None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:20.741883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bca75e-6112-4728-b53c-6275ee3306eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:20.742545 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bca75e-6112-4728-b53c-6275ee3306eb admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:20.742825 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bca75e-6112-4728-b53c-6275ee3306eb admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:20.743098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bca75e-6112-4728-b53c-6275ee3306eb admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:20.743870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bca75e-6112-4728-b53c-6275ee3306eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:36:20.744172 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bca75e-6112-4728-b53c-6275ee3306eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:20.745748 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bca75e-6112-4728-b53c-6275ee3306eb admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:20.758266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1193/2429] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:36:20.762636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973855b-eced-4d45-a50b-6b7170034e52 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:20.773620 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973855b-eced-4d45-a50b-6b7170034e52 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:20.778390 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acaf81-4632-488b-ab9a-df417c8cf3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:20.779050 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acaf81-4632-488b-ab9a-df417c8cf3c4 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:20.779360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acaf81-4632-488b-ab9a-df417c8cf3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:20.779743 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acaf81-4632-488b-ab9a-df417c8cf3c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:20.780464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48acaf81-4632-488b-ab9a-df417c8cf3c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:20.782650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48acaf81-4632-488b-ab9a-df417c8cf3c4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:20.814743 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1237/2430] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:20 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:20.815940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bca75e-6112-4728-b53c-6275ee3306eb admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:20.825812 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bca75e-6112-4728-b53c-6275ee3306eb admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:20.881021 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101d397-68ab-4f9f-8eb6-8a22a1f81702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:20.881687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101d397-68ab-4f9f-8eb6-8a22a1f81702 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:20.881966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101d397-68ab-4f9f-8eb6-8a22a1f81702 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:20.882239 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101d397-68ab-4f9f-8eb6-8a22a1f81702 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:20.882907 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101d397-68ab-4f9f-8eb6-8a22a1f81702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:20.884824 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101d397-68ab-4f9f-8eb6-8a22a1f81702 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:20.907901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1194/2431] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:20 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:36:20.909230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acaf81-4632-488b-ab9a-df417c8cf3c4 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:20.921846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acaf81-4632-488b-ab9a-df417c8cf3c4 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:20.926398 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031f401-e241-4066-8a9d-b67718db8b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:20.927058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031f401-e241-4066-8a9d-b67718db8b7b admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:20.927368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031f401-e241-4066-8a9d-b67718db8b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:20.927668 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031f401-e241-4066-8a9d-b67718db8b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:20.928380 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031f401-e241-4066-8a9d-b67718db8b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:20.930142 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031f401-e241-4066-8a9d-b67718db8b7b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:20.940457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1238/2432] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:20.944439 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101d397-68ab-4f9f-8eb6-8a22a1f81702 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:20.955129 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101d397-68ab-4f9f-8eb6-8a22a1f81702 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:20.960365 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb25e3c-dedb-43bb-bb00-51d7378ed9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:20.961036 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb25e3c-dedb-43bb-bb00-51d7378ed9da admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:20.961319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb25e3c-dedb-43bb-bb00-51d7378ed9da admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:20.961596 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb25e3c-dedb-43bb-bb00-51d7378ed9da admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:20.962327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb25e3c-dedb-43bb-bb00-51d7378ed9da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:20.964178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb25e3c-dedb-43bb-bb00-51d7378ed9da admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:20.975169 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1195/2433] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:20.979552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031f401-e241-4066-8a9d-b67718db8b7b admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:20.990218 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031f401-e241-4066-8a9d-b67718db8b7b admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:20.994856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c97a40-c63e-4223-b3e4-a433b3747355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:20.995770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97a40-c63e-4223-b3e4-a433b3747355 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:20.996315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97a40-c63e-4223-b3e4-a433b3747355 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:20.996592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97a40-c63e-4223-b3e4-a433b3747355 admin admin] PATH_INFO: `/v3/projects/a0c2d8c89875416c909c1fa7d87e947d/users/278af3b6352a454faf5452126130ab0a/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:21.011252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c97a40-c63e-4223-b3e4-a433b3747355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0c2d8c89875416c909c1fa7d87e947d, user_id=278af3b6352a454faf5452126130ab0a, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:21.013874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c97a40-c63e-4223-b3e4-a433b3747355 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:21.027718 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1239/2434] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:20 2022] PUT /identity/v3/projects/a0c2d8c89875416c909c1fa7d87e947d/users/278af3b6352a454faf5452126130ab0a/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:21.039211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4dba8-f0c6-4215-a34b-9eb9622bc8ed None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:21.039520 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4dba8-f0c6-4215-a34b-9eb9622bc8ed None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:21.039800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4dba8-f0c6-4215-a34b-9eb9622bc8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:21.067613 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-adf4dba8-f0c6-4215-a34b-9eb9622bc8ed None None] MFA Rules not processed for user `278af3b6352a454faf5452126130ab0a`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:21.068814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-adf4dba8-f0c6-4215-a34b-9eb9622bc8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:21.142485 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1196/2435] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:21.147340 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c97a40-c63e-4223-b3e4-a433b3747355 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:21.211473 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c97a40-c63e-4223-b3e4-a433b3747355 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:21.216123 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5684c8-f63a-4249-9b64-4ca9f2c029a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:21.216795 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5684c8-f63a-4249-9b64-4ca9f2c029a9 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:21.217079 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5684c8-f63a-4249-9b64-4ca9f2c029a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:21.217354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5684c8-f63a-4249-9b64-4ca9f2c029a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:21.218044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5684c8-f63a-4249-9b64-4ca9f2c029a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:21.219961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5684c8-f63a-4249-9b64-4ca9f2c029a9 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:21.246245 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1240/2436] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:21 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:36:21.250337 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4dba8-f0c6-4215-a34b-9eb9622bc8ed None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:21.261133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4dba8-f0c6-4215-a34b-9eb9622bc8ed None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:21.318404 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc633159-80d9-42d8-bcca-88b40d8f8977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:21.319081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc633159-80d9-42d8-bcca-88b40d8f8977 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:21.319394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc633159-80d9-42d8-bcca-88b40d8f8977 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:21.319686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc633159-80d9-42d8-bcca-88b40d8f8977 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:21.320377 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc633159-80d9-42d8-bcca-88b40d8f8977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:21.322309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc633159-80d9-42d8-bcca-88b40d8f8977 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:21.341958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1197/2437] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:21 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:21.345893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5684c8-f63a-4249-9b64-4ca9f2c029a9 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:21.356628 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5684c8-f63a-4249-9b64-4ca9f2c029a9 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:21.361331 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1f01d-d5da-43be-ba93-38a1e180b27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:21.361986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1f01d-d5da-43be-ba93-38a1e180b27f admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:21.362268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1f01d-d5da-43be-ba93-38a1e180b27f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:21.362548 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1f01d-d5da-43be-ba93-38a1e180b27f admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:21.363927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb1f01d-d5da-43be-ba93-38a1e180b27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:21.365506 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb1f01d-d5da-43be-ba93-38a1e180b27f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:21.377083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1241/2438] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:21.381175 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc633159-80d9-42d8-bcca-88b40d8f8977 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:21.390978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc633159-80d9-42d8-bcca-88b40d8f8977 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:21.395799 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630208d-e7f8-425f-ba86-a1aa831ff6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:21.396639 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630208d-e7f8-425f-ba86-a1aa831ff6e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:21.396923 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630208d-e7f8-425f-ba86-a1aa831ff6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:21.397192 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630208d-e7f8-425f-ba86-a1aa831ff6e0 admin admin] PATH_INFO: `/v3/projects/e2ac473b965b447b9a87d10a96f4a7d2/users/f40dcf89f12d41b18bffd8fe2dc1a643/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:21.412348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630208d-e7f8-425f-ba86-a1aa831ff6e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ac473b965b447b9a87d10a96f4a7d2, user_id=f40dcf89f12d41b18bffd8fe2dc1a643, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:21.415462 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630208d-e7f8-425f-ba86-a1aa831ff6e0 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:21.429706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1198/2439] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:21 2022] PUT /identity/v3/projects/e2ac473b965b447b9a87d10a96f4a7d2/users/f40dcf89f12d41b18bffd8fe2dc1a643/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:21.433898 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1f01d-d5da-43be-ba93-38a1e180b27f admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:21.443723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1f01d-d5da-43be-ba93-38a1e180b27f admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:21.502155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fc0ce-6716-46a7-873b-70aaf6f93da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:21.502813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fc0ce-6716-46a7-873b-70aaf6f93da1 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:21.503091 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fc0ce-6716-46a7-873b-70aaf6f93da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:21.503397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fc0ce-6716-46a7-873b-70aaf6f93da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:21.504130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fc0ce-6716-46a7-873b-70aaf6f93da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:21.505701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fc0ce-6716-46a7-873b-70aaf6f93da1 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:21.515110 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1242/2440] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:21.519866 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630208d-e7f8-425f-ba86-a1aa831ff6e0 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:21.530475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630208d-e7f8-425f-ba86-a1aa831ff6e0 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:21.535708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d383a94-b1e3-4d7e-babb-ec61ce801ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:21.536707 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d383a94-b1e3-4d7e-babb-ec61ce801ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:21.536941 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d383a94-b1e3-4d7e-babb-ec61ce801ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:21.537185 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d383a94-b1e3-4d7e-babb-ec61ce801ab1 admin admin] PATH_INFO: `/v3/projects/e2ac473b965b447b9a87d10a96f4a7d2/users/f40dcf89f12d41b18bffd8fe2dc1a643/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:21.539717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d383a94-b1e3-4d7e-babb-ec61ce801ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ac473b965b447b9a87d10a96f4a7d2, user_id=f40dcf89f12d41b18bffd8fe2dc1a643, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:21.542294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d383a94-b1e3-4d7e-babb-ec61ce801ab1 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:21.552426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1199/2441] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:21 2022] PUT /identity/v3/projects/e2ac473b965b447b9a87d10a96f4a7d2/users/f40dcf89f12d41b18bffd8fe2dc1a643/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:21.562431 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f2b39-de81-4b69-83c6-e247742774c5 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:21.562717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f2b39-de81-4b69-83c6-e247742774c5 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:21.562994 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f2b39-de81-4b69-83c6-e247742774c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:21.588274 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-274f2b39-de81-4b69-83c6-e247742774c5 None None] MFA Rules not processed for user `f40dcf89f12d41b18bffd8fe2dc1a643`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:21.589447 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-274f2b39-de81-4b69-83c6-e247742774c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:21.655918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1243/2442] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:21.668123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d383a94-b1e3-4d7e-babb-ec61ce801ab1 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:21.669896 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-2d383a94-b1e3-4d7e-babb-ec61ce801ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:21.725767 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d383a94-b1e3-4d7e-babb-ec61ce801ab1 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:21.729398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175f115-9921-48db-9387-a7eba81abe77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:21.730037 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175f115-9921-48db-9387-a7eba81abe77 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:21.730317 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175f115-9921-48db-9387-a7eba81abe77 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:21.730591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175f115-9921-48db-9387-a7eba81abe77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:21.739355 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175f115-9921-48db-9387-a7eba81abe77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:21.740763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175f115-9921-48db-9387-a7eba81abe77 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:21.750272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1200/2443] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:21.824590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f2b39-de81-4b69-83c6-e247742774c5 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:21.835589 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f2b39-de81-4b69-83c6-e247742774c5 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:21.840158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0139aa2-1ecb-456a-a089-58783465fcbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:21.840850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0139aa2-1ecb-456a-a089-58783465fcbf service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:21.841129 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0139aa2-1ecb-456a-a089-58783465fcbf service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:21.841568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0139aa2-1ecb-456a-a089-58783465fcbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:21.900381 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0139aa2-1ecb-456a-a089-58783465fcbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:21.901713 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0139aa2-1ecb-456a-a089-58783465fcbf service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:21.910514 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1244/2444] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:31.705819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175f115-9921-48db-9387-a7eba81abe77 service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:31.765195 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175f115-9921-48db-9387-a7eba81abe77 service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:31.769709 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba88c8f-908e-4ca9-9b08-53571adebe09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:31.770566 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba88c8f-908e-4ca9-9b08-53571adebe09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:31.770850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba88c8f-908e-4ca9-9b08-53571adebe09 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:31.771128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba88c8f-908e-4ca9-9b08-53571adebe09 admin admin] PATH_INFO: `/v3/users/278af3b6352a454faf5452126130ab0a` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:31.772326 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba88c8f-908e-4ca9-9b08-53571adebe09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=278af3b6352a454faf5452126130ab0a)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:31.774221 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba88c8f-908e-4ca9-9b08-53571adebe09 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:31.817668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-cba88c8f-908e-4ca9-9b08-53571adebe09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '278af3b6352a454faf5452126130ab0a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:31.824393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-cba88c8f-908e-4ca9-9b08-53571adebe09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '278af3b6352a454faf5452126130ab0a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:31.825050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-cba88c8f-908e-4ca9-9b08-53571adebe09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '278af3b6352a454faf5452126130ab0a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:31.831968 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-cba88c8f-908e-4ca9-9b08-53571adebe09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '278af3b6352a454faf5452126130ab0a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:31.832586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-cba88c8f-908e-4ca9-9b08-53571adebe09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '278af3b6352a454faf5452126130ab0a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:31.848730 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1201/2445] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:31 2022] DELETE /identity/v3/users/278af3b6352a454faf5452126130ab0a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:31.854141 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212acc42-976a-4d7f-b5d4-273a629f585b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:31.854607 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212acc42-976a-4d7f-b5d4-273a629f585b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:31.854704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212acc42-976a-4d7f-b5d4-273a629f585b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:31.871901 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-212acc42-976a-4d7f-b5d4-273a629f585b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:31.873067 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-212acc42-976a-4d7f-b5d4-273a629f585b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:31.932461 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1245/2446] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:31.941871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba88c8f-908e-4ca9-9b08-53571adebe09 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:31.943553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-cba88c8f-908e-4ca9-9b08-53571adebe09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:32.004441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba88c8f-908e-4ca9-9b08-53571adebe09 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:32.008969 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a514381-1d13-4e6b-adb8-5589c4452f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:32.009723 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a514381-1d13-4e6b-adb8-5589c4452f66 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:32.010007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a514381-1d13-4e6b-adb8-5589c4452f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:32.010404 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a514381-1d13-4e6b-adb8-5589c4452f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:32.021254 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a514381-1d13-4e6b-adb8-5589c4452f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:32.023268 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a514381-1d13-4e6b-adb8-5589c4452f66 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:32.033066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1202/2447] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:32.325377 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212acc42-976a-4d7f-b5d4-273a629f585b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:32.326873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-212acc42-976a-4d7f-b5d4-273a629f585b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:32.339790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212acc42-976a-4d7f-b5d4-273a629f585b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:32.344841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:32.345872 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:32.346230 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:32.346571 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] PATH_INFO: `/v3/projects/a0c2d8c89875416c909c1fa7d87e947d` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:32.347844 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0c2d8c89875416c909c1fa7d87e947d)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:32.349965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:32.411730 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0c2d8c89875416c909c1fa7d87e947d', 'request_id': 'req-a35762b0-1ce9-40ea-b500-1b014b226096'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:32.416963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0c2d8c89875416c909c1fa7d87e947d', 'request_id': 'req-a35762b0-1ce9-40ea-b500-1b014b226096'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:32.424249 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0c2d8c89875416c909c1fa7d87e947d', 'request_id': 'req-a35762b0-1ce9-40ea-b500-1b014b226096'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:32.435331 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] The token cache is being invalidate because project a0c2d8c89875416c909c1fa7d87e947d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:32.435621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:32.436839 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:32.438407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1246/2448] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:32 2022] DELETE /identity/v3/projects/a0c2d8c89875416c909c1fa7d87e947d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:32.442367 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a514381-1d13-4e6b-adb8-5589c4452f66 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:32.452292 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a514381-1d13-4e6b-adb8-5589c4452f66 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:32.508617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be11789-0c77-4ca5-8041-508ba94fca6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:32.509497 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be11789-0c77-4ca5-8041-508ba94fca6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:32.509778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be11789-0c77-4ca5-8041-508ba94fca6e admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:32.510053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be11789-0c77-4ca5-8041-508ba94fca6e admin admin] PATH_INFO: `/v3/users/f40dcf89f12d41b18bffd8fe2dc1a643` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:32.511295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be11789-0c77-4ca5-8041-508ba94fca6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f40dcf89f12d41b18bffd8fe2dc1a643)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:32.513216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be11789-0c77-4ca5-8041-508ba94fca6e admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:32.558783 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7be11789-0c77-4ca5-8041-508ba94fca6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f40dcf89f12d41b18bffd8fe2dc1a643'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:32.565604 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7be11789-0c77-4ca5-8041-508ba94fca6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f40dcf89f12d41b18bffd8fe2dc1a643'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:32.566872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7be11789-0c77-4ca5-8041-508ba94fca6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f40dcf89f12d41b18bffd8fe2dc1a643'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:32.574990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7be11789-0c77-4ca5-8041-508ba94fca6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f40dcf89f12d41b18bffd8fe2dc1a643'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:32.575674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7be11789-0c77-4ca5-8041-508ba94fca6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f40dcf89f12d41b18bffd8fe2dc1a643'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:32.594510 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1203/2449] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:32 2022] DELETE /identity/v3/users/f40dcf89f12d41b18bffd8fe2dc1a643 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:32.874156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:32.875813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:32.890507 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35762b0-1ce9-40ea-b500-1b014b226096 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:32.952672 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a851ab1-9b33-4688-8652-aab8a3d18db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:32.953551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a851ab1-9b33-4688-8652-aab8a3d18db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:32.953832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a851ab1-9b33-4688-8652-aab8a3d18db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:32.954866 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a851ab1-9b33-4688-8652-aab8a3d18db2 admin admin] PATH_INFO: `/v3/projects/e2ac473b965b447b9a87d10a96f4a7d2` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:32.961814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a851ab1-9b33-4688-8652-aab8a3d18db2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2ac473b965b447b9a87d10a96f4a7d2)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:32.961814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a851ab1-9b33-4688-8652-aab8a3d18db2 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:33.008628 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5a851ab1-9b33-4688-8652-aab8a3d18db2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2ac473b965b447b9a87d10a96f4a7d2', 'request_id': 'req-5a851ab1-9b33-4688-8652-aab8a3d18db2'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:33.013957 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5a851ab1-9b33-4688-8652-aab8a3d18db2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2ac473b965b447b9a87d10a96f4a7d2', 'request_id': 'req-5a851ab1-9b33-4688-8652-aab8a3d18db2'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:33.021713 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5a851ab1-9b33-4688-8652-aab8a3d18db2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2ac473b965b447b9a87d10a96f4a7d2', 'request_id': 'req-5a851ab1-9b33-4688-8652-aab8a3d18db2'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:33.033373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5a851ab1-9b33-4688-8652-aab8a3d18db2 admin admin] The token cache is being invalidate because project e2ac473b965b447b9a87d10a96f4a7d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:33.033669 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5a851ab1-9b33-4688-8652-aab8a3d18db2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:33.034321 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-5a851ab1-9b33-4688-8652-aab8a3d18db2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:33.036060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1247/2450] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:32 2022] DELETE /identity/v3/projects/e2ac473b965b447b9a87d10a96f4a7d2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:33.040941 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0a54b-38ef-4fbf-aaea-9263f7abca80 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:33.041456 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0a54b-38ef-4fbf-aaea-9263f7abca80 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:33.041554 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0a54b-38ef-4fbf-aaea-9263f7abca80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:33.059570 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-4dd0a54b-38ef-4fbf-aaea-9263f7abca80 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:33.060803 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-4dd0a54b-38ef-4fbf-aaea-9263f7abca80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:33.122060 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1204/2451] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:33.131214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb3eed-7f44-4ae2-9333-3fa2ea0e8483 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:33.132316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb3eed-7f44-4ae2-9333-3fa2ea0e8483 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:33.132900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb3eed-7f44-4ae2-9333-3fa2ea0e8483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:33.152987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-34fb3eed-7f44-4ae2-9333-3fa2ea0e8483 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:33.154201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-34fb3eed-7f44-4ae2-9333-3fa2ea0e8483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:33.162805 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1248/2452] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:33.167143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0a54b-38ef-4fbf-aaea-9263f7abca80 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:33.176942 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0a54b-38ef-4fbf-aaea-9263f7abca80 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:33.181801 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302129fc-9aea-45a2-bec1-57d920698b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:33.182461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302129fc-9aea-45a2-bec1-57d920698b6f admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:33.182744 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302129fc-9aea-45a2-bec1-57d920698b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:33.183022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302129fc-9aea-45a2-bec1-57d920698b6f admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:33.183776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302129fc-9aea-45a2-bec1-57d920698b6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:36:33.184084 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302129fc-9aea-45a2-bec1-57d920698b6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:33.185782 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302129fc-9aea-45a2-bec1-57d920698b6f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:33.198274 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1205/2453] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:36:33.202388 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb3eed-7f44-4ae2-9333-3fa2ea0e8483 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:33.212693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb3eed-7f44-4ae2-9333-3fa2ea0e8483 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:33.217682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23308b43-afab-4c78-89b7-3e3f8066992c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:33.218457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23308b43-afab-4c78-89b7-3e3f8066992c admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:33.218748 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23308b43-afab-4c78-89b7-3e3f8066992c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:33.220358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23308b43-afab-4c78-89b7-3e3f8066992c admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:33.220358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23308b43-afab-4c78-89b7-3e3f8066992c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:33.221738 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23308b43-afab-4c78-89b7-3e3f8066992c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:33.251220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1249/2454] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:33 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:33.255710 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302129fc-9aea-45a2-bec1-57d920698b6f admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:33.268521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302129fc-9aea-45a2-bec1-57d920698b6f admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:33.310975 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23308b43-afab-4c78-89b7-3e3f8066992c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:33.313355 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-23308b43-afab-4c78-89b7-3e3f8066992c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:33.344747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6b427-93dd-4cb5-8ae7-c592a2f11733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:33.346104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6b427-93dd-4cb5-8ae7-c592a2f11733 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:33.346895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6b427-93dd-4cb5-8ae7-c592a2f11733 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:33.347589 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6b427-93dd-4cb5-8ae7-c592a2f11733 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:33.349229 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6b427-93dd-4cb5-8ae7-c592a2f11733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:33.351403 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6b427-93dd-4cb5-8ae7-c592a2f11733 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:33.385490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23308b43-afab-4c78-89b7-3e3f8066992c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:33.390056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0e28e-8d2a-444f-98d4-37048598af95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:33.390821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0e28e-8d2a-444f-98d4-37048598af95 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:33.391203 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0e28e-8d2a-444f-98d4-37048598af95 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:33.391555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0e28e-8d2a-444f-98d4-37048598af95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:33.392616 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1206/2455] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:33 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:33.393697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-eda0e28e-8d2a-444f-98d4-37048598af95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:33.399360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6b427-93dd-4cb5-8ae7-c592a2f11733 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:33.415774 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6b427-93dd-4cb5-8ae7-c592a2f11733 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:33.421876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74a39e-d160-4816-9ce7-444c28a5b904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:33.422563 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74a39e-d160-4816-9ce7-444c28a5b904 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:33.423013 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74a39e-d160-4816-9ce7-444c28a5b904 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:33.423389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74a39e-d160-4816-9ce7-444c28a5b904 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:33.424193 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74a39e-d160-4816-9ce7-444c28a5b904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:33.425952 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74a39e-d160-4816-9ce7-444c28a5b904 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:33.458441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1207/2456] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:33.460306 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74a39e-d160-4816-9ce7-444c28a5b904 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:33.471988 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74a39e-d160-4816-9ce7-444c28a5b904 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:33.475247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0e28e-8d2a-444f-98d4-37048598af95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:33.477058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c7b29-cea4-4d39-acc5-de848760e8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:33.477436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0e28e-8d2a-444f-98d4-37048598af95 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:33.477738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c7b29-cea4-4d39-acc5-de848760e8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:33.478521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c7b29-cea4-4d39-acc5-de848760e8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:33.478802 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c7b29-cea4-4d39-acc5-de848760e8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:33.479586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c7b29-cea4-4d39-acc5-de848760e8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:33.481458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c7b29-cea4-4d39-acc5-de848760e8c4 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:33.495070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1250/2457] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:36:33.499323 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1208/2458] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:33.505315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0e28e-8d2a-444f-98d4-37048598af95 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:33.523678 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0e28e-8d2a-444f-98d4-37048598af95 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:33.529597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad99bae-ee68-426f-aefd-d470aeaab4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:33.530481 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad99bae-ee68-426f-aefd-d470aeaab4b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:33.530767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad99bae-ee68-426f-aefd-d470aeaab4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:33.531043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad99bae-ee68-426f-aefd-d470aeaab4b2 admin admin] PATH_INFO: `/v3/projects/5a5dad103a6b420cab5b571325f2297e/users/cd5c62b85aac4eb29d07f0fe17f2b021/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:33.552035 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad99bae-ee68-426f-aefd-d470aeaab4b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a5dad103a6b420cab5b571325f2297e, user_id=cd5c62b85aac4eb29d07f0fe17f2b021, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:33.554800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad99bae-ee68-426f-aefd-d470aeaab4b2 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:33.571392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1251/2459] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:33 2022] PUT /identity/v3/projects/5a5dad103a6b420cab5b571325f2297e/users/cd5c62b85aac4eb29d07f0fe17f2b021/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:33.582822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bec6dd-354d-4fd3-a505-921f05315b73 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:33.583161 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bec6dd-354d-4fd3-a505-921f05315b73 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:33.583490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bec6dd-354d-4fd3-a505-921f05315b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:33.614064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-34bec6dd-354d-4fd3-a505-921f05315b73 None None] MFA Rules not processed for user `cd5c62b85aac4eb29d07f0fe17f2b021`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:33.615386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-34bec6dd-354d-4fd3-a505-921f05315b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:33.692109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1209/2460] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:33.696624 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad99bae-ee68-426f-aefd-d470aeaab4b2 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:33.774087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad99bae-ee68-426f-aefd-d470aeaab4b2 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:33.779060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c193535-300b-42ce-9b11-cb3c7833f7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:33.780115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c193535-300b-42ce-9b11-cb3c7833f7d4 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:33.780472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c193535-300b-42ce-9b11-cb3c7833f7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:33.780816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c193535-300b-42ce-9b11-cb3c7833f7d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:33.781578 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c193535-300b-42ce-9b11-cb3c7833f7d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:33.783580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c193535-300b-42ce-9b11-cb3c7833f7d4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:33.815958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1252/2461] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:33 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:36:33.817377 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bec6dd-354d-4fd3-a505-921f05315b73 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:33.830278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bec6dd-354d-4fd3-a505-921f05315b73 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:33.923021 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be4b33-9557-4a42-8ed1-e2b9a8699766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:33.923767 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be4b33-9557-4a42-8ed1-e2b9a8699766 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:33.924117 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be4b33-9557-4a42-8ed1-e2b9a8699766 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:33.924416 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be4b33-9557-4a42-8ed1-e2b9a8699766 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:33.925211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be4b33-9557-4a42-8ed1-e2b9a8699766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:33.927375 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be4b33-9557-4a42-8ed1-e2b9a8699766 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:33.968314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1210/2462] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:33 2022] POST /identity/v3/users => generated 299 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:33.973232 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c193535-300b-42ce-9b11-cb3c7833f7d4 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:33.984077 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c193535-300b-42ce-9b11-cb3c7833f7d4 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:33.989705 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7f5f1-9b17-435f-a194-23c5cfc1003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:33.990379 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7f5f1-9b17-435f-a194-23c5cfc1003a admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:33.991114 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7f5f1-9b17-435f-a194-23c5cfc1003a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:33.991114 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7f5f1-9b17-435f-a194-23c5cfc1003a admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:33.991882 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7f5f1-9b17-435f-a194-23c5cfc1003a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:33.993498 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7f5f1-9b17-435f-a194-23c5cfc1003a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:34.013153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1253/2463] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:34.017188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be4b33-9557-4a42-8ed1-e2b9a8699766 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:34.028792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be4b33-9557-4a42-8ed1-e2b9a8699766 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:34.033713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937222c-b26d-4d6c-ac3e-4cf7a6ddaaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:34.034676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937222c-b26d-4d6c-ac3e-4cf7a6ddaaf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:34.034962 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937222c-b26d-4d6c-ac3e-4cf7a6ddaaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:34.035305 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937222c-b26d-4d6c-ac3e-4cf7a6ddaaf4 admin admin] PATH_INFO: `/v3/projects/128ac48cd3324453a0e9387f21cfc9bd/users/0966697d4982453d8df94cf61f892525/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:34.050591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937222c-b26d-4d6c-ac3e-4cf7a6ddaaf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=128ac48cd3324453a0e9387f21cfc9bd, user_id=0966697d4982453d8df94cf61f892525, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:34.053306 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937222c-b26d-4d6c-ac3e-4cf7a6ddaaf4 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:34.070045 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1211/2464] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:34 2022] PUT /identity/v3/projects/128ac48cd3324453a0e9387f21cfc9bd/users/0966697d4982453d8df94cf61f892525/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:34.080484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7f5f1-9b17-435f-a194-23c5cfc1003a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:34.091882 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7f5f1-9b17-435f-a194-23c5cfc1003a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:34.156448 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79dce5-14aa-4b0a-b38f-275e0540da06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:34.157157 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79dce5-14aa-4b0a-b38f-275e0540da06 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:34.157464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79dce5-14aa-4b0a-b38f-275e0540da06 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:34.157765 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79dce5-14aa-4b0a-b38f-275e0540da06 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:34.158502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79dce5-14aa-4b0a-b38f-275e0540da06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:34.160138 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79dce5-14aa-4b0a-b38f-275e0540da06 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:34.171206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1254/2465] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:34.175547 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937222c-b26d-4d6c-ac3e-4cf7a6ddaaf4 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:34.188577 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937222c-b26d-4d6c-ac3e-4cf7a6ddaaf4 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:34.193890 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf613b-6d58-4c2a-b93e-70a2c4140bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:34.195578 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf613b-6d58-4c2a-b93e-70a2c4140bab admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:34.196071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf613b-6d58-4c2a-b93e-70a2c4140bab admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:34.196348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf613b-6d58-4c2a-b93e-70a2c4140bab admin admin] PATH_INFO: `/v3/projects/128ac48cd3324453a0e9387f21cfc9bd/users/0966697d4982453d8df94cf61f892525/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:34.198523 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf613b-6d58-4c2a-b93e-70a2c4140bab admin admin] RBAC: Authorizing `identity:create_grant(project_id=128ac48cd3324453a0e9387f21cfc9bd, user_id=0966697d4982453d8df94cf61f892525, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:34.201174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf613b-6d58-4c2a-b93e-70a2c4140bab admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:34.227452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1212/2466] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:34 2022] PUT /identity/v3/projects/128ac48cd3324453a0e9387f21cfc9bd/users/0966697d4982453d8df94cf61f892525/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:34.233544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f937953-d850-4d00-8504-2aa5b07ace05 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:34.233885 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f937953-d850-4d00-8504-2aa5b07ace05 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:34.234069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f937953-d850-4d00-8504-2aa5b07ace05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:34.272623 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9f937953-d850-4d00-8504-2aa5b07ace05 None None] MFA Rules not processed for user `0966697d4982453d8df94cf61f892525`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:34.273864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9f937953-d850-4d00-8504-2aa5b07ace05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:34.349439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1255/2467] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:34.361109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf613b-6d58-4c2a-b93e-70a2c4140bab admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:34.363264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-efbf613b-6d58-4c2a-b93e-70a2c4140bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:34.424727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf613b-6d58-4c2a-b93e-70a2c4140bab admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:34.429820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25732e85-a34a-44a2-91e2-d764c04676ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:34.430831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25732e85-a34a-44a2-91e2-d764c04676ce service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:34.431310 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25732e85-a34a-44a2-91e2-d764c04676ce service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:34.431899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25732e85-a34a-44a2-91e2-d764c04676ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:34.442082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25732e85-a34a-44a2-91e2-d764c04676ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:34.443590 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25732e85-a34a-44a2-91e2-d764c04676ce service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:34.455808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1213/2468] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:36:34.485103 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f937953-d850-4d00-8504-2aa5b07ace05 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:34.561833 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f937953-d850-4d00-8504-2aa5b07ace05 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:34.567666 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ee2a4-f7e5-44f4-bc21-ce58191a1ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:34.569244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ee2a4-f7e5-44f4-bc21-ce58191a1ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:34.569758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ee2a4-f7e5-44f4-bc21-ce58191a1ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:34.570239 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ee2a4-f7e5-44f4-bc21-ce58191a1ac0 admin admin] PATH_INFO: `/v3/users/cd5c62b85aac4eb29d07f0fe17f2b021` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:34.571722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ee2a4-f7e5-44f4-bc21-ce58191a1ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd5c62b85aac4eb29d07f0fe17f2b021)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:34.574124 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ee2a4-f7e5-44f4-bc21-ce58191a1ac0 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:34.632713 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d57ee2a4-f7e5-44f4-bc21-ce58191a1ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd5c62b85aac4eb29d07f0fe17f2b021'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:34.635459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d57ee2a4-f7e5-44f4-bc21-ce58191a1ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd5c62b85aac4eb29d07f0fe17f2b021'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:34.647722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d57ee2a4-f7e5-44f4-bc21-ce58191a1ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd5c62b85aac4eb29d07f0fe17f2b021'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:34.657819 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d57ee2a4-f7e5-44f4-bc21-ce58191a1ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd5c62b85aac4eb29d07f0fe17f2b021'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:34.658593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-d57ee2a4-f7e5-44f4-bc21-ce58191a1ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd5c62b85aac4eb29d07f0fe17f2b021'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:34.686261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1256/2469] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:34 2022] DELETE /identity/v3/users/cd5c62b85aac4eb29d07f0fe17f2b021 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:34.688207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71c48f-afe8-4113-b59a-0e7bbaf49a59 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:34.688612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71c48f-afe8-4113-b59a-0e7bbaf49a59 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:34.692180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71c48f-afe8-4113-b59a-0e7bbaf49a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:34.710726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-7a71c48f-afe8-4113-b59a-0e7bbaf49a59 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:34.711977 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7a71c48f-afe8-4113-b59a-0e7bbaf49a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:34.771952 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1214/2470] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:36:34.782164 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ee2a4-f7e5-44f4-bc21-ce58191a1ac0 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:34.783651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-d57ee2a4-f7e5-44f4-bc21-ce58191a1ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:34.842911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ee2a4-f7e5-44f4-bc21-ce58191a1ac0 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:34.846756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b3980-c996-487b-b22f-4bef35cd80ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:34.847435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b3980-c996-487b-b22f-4bef35cd80ba service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:34.847721 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b3980-c996-487b-b22f-4bef35cd80ba service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:34.847998 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b3980-c996-487b-b22f-4bef35cd80ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:34.860229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b3980-c996-487b-b22f-4bef35cd80ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:34.861832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b3980-c996-487b-b22f-4bef35cd80ba service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:34.873060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1257/2471] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:35.149400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71c48f-afe8-4113-b59a-0e7bbaf49a59 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:35.151077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7a71c48f-afe8-4113-b59a-0e7bbaf49a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:35.163901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71c48f-afe8-4113-b59a-0e7bbaf49a59 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:35.168958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:35.170047 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:35.170670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:35.171097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] PATH_INFO: `/v3/projects/5a5dad103a6b420cab5b571325f2297e` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:35.172490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a5dad103a6b420cab5b571325f2297e)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:35.174540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:35.237383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a5dad103a6b420cab5b571325f2297e', 'request_id': 'req-6f09515a-07b6-4734-96ee-0bb6a51de176'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.245213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a5dad103a6b420cab5b571325f2297e', 'request_id': 'req-6f09515a-07b6-4734-96ee-0bb6a51de176'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.274781 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a5dad103a6b420cab5b571325f2297e', 'request_id': 'req-6f09515a-07b6-4734-96ee-0bb6a51de176'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.295499 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] The token cache is being invalidate because project 5a5dad103a6b420cab5b571325f2297e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:35.295858 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.296617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.298345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1215/2472] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:35 2022] DELETE /identity/v3/projects/5a5dad103a6b420cab5b571325f2297e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:35.302162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b3980-c996-487b-b22f-4bef35cd80ba service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:35.313575 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b3980-c996-487b-b22f-4bef35cd80ba service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:35.381830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff38d5b-23b0-4d1b-93c3-f1c1b1d3c6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:35.382802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff38d5b-23b0-4d1b-93c3-f1c1b1d3c6e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:35.383151 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff38d5b-23b0-4d1b-93c3-f1c1b1d3c6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:35.383515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff38d5b-23b0-4d1b-93c3-f1c1b1d3c6e3 admin admin] PATH_INFO: `/v3/users/0966697d4982453d8df94cf61f892525` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:35.385282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff38d5b-23b0-4d1b-93c3-f1c1b1d3c6e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0966697d4982453d8df94cf61f892525)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:35.387358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff38d5b-23b0-4d1b-93c3-f1c1b1d3c6e3 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:35.437281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1ff38d5b-23b0-4d1b-93c3-f1c1b1d3c6e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0966697d4982453d8df94cf61f892525'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.438125 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1ff38d5b-23b0-4d1b-93c3-f1c1b1d3c6e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0966697d4982453d8df94cf61f892525'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.447455 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1ff38d5b-23b0-4d1b-93c3-f1c1b1d3c6e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0966697d4982453d8df94cf61f892525'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.455886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1ff38d5b-23b0-4d1b-93c3-f1c1b1d3c6e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0966697d4982453d8df94cf61f892525'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.456623 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-1ff38d5b-23b0-4d1b-93c3-f1c1b1d3c6e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0966697d4982453d8df94cf61f892525'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.483323 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1258/2473] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:35 2022] DELETE /identity/v3/users/0966697d4982453d8df94cf61f892525 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:35.789357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:35.791360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:35.809554 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09515a-07b6-4734-96ee-0bb6a51de176 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:35.881214 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440b32e-8881-487f-a91c-8e19a54ee89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:35.881798 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440b32e-8881-487f-a91c-8e19a54ee89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:35.882087 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440b32e-8881-487f-a91c-8e19a54ee89a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:35.882368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440b32e-8881-487f-a91c-8e19a54ee89a admin admin] PATH_INFO: `/v3/projects/128ac48cd3324453a0e9387f21cfc9bd` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:35.883643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440b32e-8881-487f-a91c-8e19a54ee89a admin admin] RBAC: Authorizing `identity:delete_project(project_id=128ac48cd3324453a0e9387f21cfc9bd)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:35.885664 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440b32e-8881-487f-a91c-8e19a54ee89a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:35.938041 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-5440b32e-8881-487f-a91c-8e19a54ee89a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '128ac48cd3324453a0e9387f21cfc9bd', 'request_id': 'req-5440b32e-8881-487f-a91c-8e19a54ee89a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.945255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-5440b32e-8881-487f-a91c-8e19a54ee89a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '128ac48cd3324453a0e9387f21cfc9bd', 'request_id': 'req-5440b32e-8881-487f-a91c-8e19a54ee89a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.954026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-5440b32e-8881-487f-a91c-8e19a54ee89a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '128ac48cd3324453a0e9387f21cfc9bd', 'request_id': 'req-5440b32e-8881-487f-a91c-8e19a54ee89a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.967341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-5440b32e-8881-487f-a91c-8e19a54ee89a admin admin] The token cache is being invalidate because project 128ac48cd3324453a0e9387f21cfc9bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:35.967634 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-5440b32e-8881-487f-a91c-8e19a54ee89a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.968312 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-5440b32e-8881-487f-a91c-8e19a54ee89a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:35.973098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1216/2474] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:35 2022] DELETE /identity/v3/projects/128ac48cd3324453a0e9387f21cfc9bd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:35.975138 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb497168-6699-4c1f-a71f-5ae1f1c84d88 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:35.976193 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb497168-6699-4c1f-a71f-5ae1f1c84d88 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:35.976561 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb497168-6699-4c1f-a71f-5ae1f1c84d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:35.996261 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-eb497168-6699-4c1f-a71f-5ae1f1c84d88 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:35.997592 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-eb497168-6699-4c1f-a71f-5ae1f1c84d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:36.062856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1259/2475] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:36:36.072165 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbf750-6c78-4636-93e0-4482bcce358a None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.072455 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbf750-6c78-4636-93e0-4482bcce358a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.072736 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbf750-6c78-4636-93e0-4482bcce358a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.091814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-07dbf750-6c78-4636-93e0-4482bcce358a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:36.092966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-07dbf750-6c78-4636-93e0-4482bcce358a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:36.102077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1217/2476] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:36.106348 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb497168-6699-4c1f-a71f-5ae1f1c84d88 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:36.117412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb497168-6699-4c1f-a71f-5ae1f1c84d88 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:36.122435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef61cd9-3ad4-4044-8844-cd945fcba99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:36.123105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef61cd9-3ad4-4044-8844-cd945fcba99a admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.123423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef61cd9-3ad4-4044-8844-cd945fcba99a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.124070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef61cd9-3ad4-4044-8844-cd945fcba99a admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.124427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef61cd9-3ad4-4044-8844-cd945fcba99a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:36:36.124738 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef61cd9-3ad4-4044-8844-cd945fcba99a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:36.127287 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef61cd9-3ad4-4044-8844-cd945fcba99a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:36.145166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1260/2477] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:36:36.149720 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbf750-6c78-4636-93e0-4482bcce358a None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:36.159085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbf750-6c78-4636-93e0-4482bcce358a None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:36.164296 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76456f7-7694-4be9-bb75-e18602e6ce02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:36.165001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76456f7-7694-4be9-bb75-e18602e6ce02 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.165286 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76456f7-7694-4be9-bb75-e18602e6ce02 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.165557 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76456f7-7694-4be9-bb75-e18602e6ce02 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.166244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76456f7-7694-4be9-bb75-e18602e6ce02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:36.168190 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76456f7-7694-4be9-bb75-e18602e6ce02 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:36.199605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1218/2478] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:36 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:36.203854 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef61cd9-3ad4-4044-8844-cd945fcba99a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:36.214072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef61cd9-3ad4-4044-8844-cd945fcba99a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:36.287579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d1b04-4d29-4573-85ea-7a10f0c0fd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:36.288806 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d1b04-4d29-4573-85ea-7a10f0c0fd44 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.289718 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d1b04-4d29-4573-85ea-7a10f0c0fd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.289996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d1b04-4d29-4573-85ea-7a10f0c0fd44 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.290856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d1b04-4d29-4573-85ea-7a10f0c0fd44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:36.292998 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d1b04-4d29-4573-85ea-7a10f0c0fd44 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:36.317025 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1261/2479] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:36 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:36.318892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76456f7-7694-4be9-bb75-e18602e6ce02 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:36.329412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76456f7-7694-4be9-bb75-e18602e6ce02 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:36.334037 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0742bac-4a49-41b5-a94d-e48f9d63fded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:36.334756 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0742bac-4a49-41b5-a94d-e48f9d63fded admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.335035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0742bac-4a49-41b5-a94d-e48f9d63fded admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.335337 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0742bac-4a49-41b5-a94d-e48f9d63fded admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.336064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0742bac-4a49-41b5-a94d-e48f9d63fded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:36.337619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0742bac-4a49-41b5-a94d-e48f9d63fded admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:36.348390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1219/2480] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:36.352799 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d1b04-4d29-4573-85ea-7a10f0c0fd44 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:36.362385 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d1b04-4d29-4573-85ea-7a10f0c0fd44 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:36.367507 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96de2d-03df-433d-b4e2-1c5c7ae08479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:36.368148 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96de2d-03df-433d-b4e2-1c5c7ae08479 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.368424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96de2d-03df-433d-b4e2-1c5c7ae08479 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.368693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96de2d-03df-433d-b4e2-1c5c7ae08479 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.369394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96de2d-03df-433d-b4e2-1c5c7ae08479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:36.371229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96de2d-03df-433d-b4e2-1c5c7ae08479 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:36.380648 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1262/2481] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:36.384701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0742bac-4a49-41b5-a94d-e48f9d63fded admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:36.394299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0742bac-4a49-41b5-a94d-e48f9d63fded admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:36.399402 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d422a5-5ccf-495f-937f-645bd8f7a2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:36.400262 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d422a5-5ccf-495f-937f-645bd8f7a2c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.400544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d422a5-5ccf-495f-937f-645bd8f7a2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.400820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d422a5-5ccf-495f-937f-645bd8f7a2c3 admin admin] PATH_INFO: `/v3/projects/3b5b0552533f45a197bbb74d4edc9ae9/users/6082d27a844b41a192d546200cf9521c/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.415709 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d422a5-5ccf-495f-937f-645bd8f7a2c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b5b0552533f45a197bbb74d4edc9ae9, user_id=6082d27a844b41a192d546200cf9521c, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:36.418426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d422a5-5ccf-495f-937f-645bd8f7a2c3 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:36.431538 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1220/2482] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:36 2022] PUT /identity/v3/projects/3b5b0552533f45a197bbb74d4edc9ae9/users/6082d27a844b41a192d546200cf9521c/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:36.441899 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f444ab-b38b-45a5-a6ac-99c8d574557a None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.442208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f444ab-b38b-45a5-a6ac-99c8d574557a None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.442607 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f444ab-b38b-45a5-a6ac-99c8d574557a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.468779 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-31f444ab-b38b-45a5-a6ac-99c8d574557a None None] MFA Rules not processed for user `6082d27a844b41a192d546200cf9521c`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:36.469970 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-31f444ab-b38b-45a5-a6ac-99c8d574557a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:36.535502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1263/2483] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:36.540185 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d422a5-5ccf-495f-937f-645bd8f7a2c3 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:36.612384 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d422a5-5ccf-495f-937f-645bd8f7a2c3 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:36.616927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b8c63-46ba-4a0f-953d-bd8ae3e5182d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:36.617591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b8c63-46ba-4a0f-953d-bd8ae3e5182d admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.617873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b8c63-46ba-4a0f-953d-bd8ae3e5182d admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.618145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b8c63-46ba-4a0f-953d-bd8ae3e5182d admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.618822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6b8c63-46ba-4a0f-953d-bd8ae3e5182d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:36.621008 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6b8c63-46ba-4a0f-953d-bd8ae3e5182d admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:36.647026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1221/2484] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:36 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:36.651680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f444ab-b38b-45a5-a6ac-99c8d574557a None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:36.662486 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f444ab-b38b-45a5-a6ac-99c8d574557a None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:36.719952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f85b7-0875-45a2-825e-1405ae53c6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:36.720637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f85b7-0875-45a2-825e-1405ae53c6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.720948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f85b7-0875-45a2-825e-1405ae53c6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.721223 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f85b7-0875-45a2-825e-1405ae53c6c9 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.721982 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f85b7-0875-45a2-825e-1405ae53c6c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:36.723952 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f85b7-0875-45a2-825e-1405ae53c6c9 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:36.744061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1264/2485] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:36 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:36.748413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b8c63-46ba-4a0f-953d-bd8ae3e5182d admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:36.758815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b8c63-46ba-4a0f-953d-bd8ae3e5182d admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:36.763405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f012d7c-9e85-4ae9-90f7-c2d2a25d28b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:36.764062 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f012d7c-9e85-4ae9-90f7-c2d2a25d28b5 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.764342 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f012d7c-9e85-4ae9-90f7-c2d2a25d28b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.764628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f012d7c-9e85-4ae9-90f7-c2d2a25d28b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.765340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f012d7c-9e85-4ae9-90f7-c2d2a25d28b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:36.766912 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f012d7c-9e85-4ae9-90f7-c2d2a25d28b5 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:36.777909 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1222/2486] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:36.782041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f85b7-0875-45a2-825e-1405ae53c6c9 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:36.792317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f85b7-0875-45a2-825e-1405ae53c6c9 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:36.796864 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f567bf-8aeb-4bc2-93ed-f5e36c9348d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:36.797734 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f567bf-8aeb-4bc2-93ed-f5e36c9348d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.798013 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f567bf-8aeb-4bc2-93ed-f5e36c9348d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.798286 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f567bf-8aeb-4bc2-93ed-f5e36c9348d4 admin admin] PATH_INFO: `/v3/projects/f87417e1032f44428a44848612c06297/users/a27725e89ee74ab39c275e9e3e790540/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.812607 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f567bf-8aeb-4bc2-93ed-f5e36c9348d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87417e1032f44428a44848612c06297, user_id=a27725e89ee74ab39c275e9e3e790540, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:36.815246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f567bf-8aeb-4bc2-93ed-f5e36c9348d4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:36.828951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1265/2487] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:36 2022] PUT /identity/v3/projects/f87417e1032f44428a44848612c06297/users/a27725e89ee74ab39c275e9e3e790540/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:36.833199 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f012d7c-9e85-4ae9-90f7-c2d2a25d28b5 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:36.843977 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f012d7c-9e85-4ae9-90f7-c2d2a25d28b5 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:36.899452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8aed7e-be70-406b-a400-119d24520cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:36.900127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8aed7e-be70-406b-a400-119d24520cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.900415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8aed7e-be70-406b-a400-119d24520cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.900684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8aed7e-be70-406b-a400-119d24520cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.901395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8aed7e-be70-406b-a400-119d24520cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:36.902967 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8aed7e-be70-406b-a400-119d24520cc7 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:36.915964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1223/2488] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:36.917790 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f567bf-8aeb-4bc2-93ed-f5e36c9348d4 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:36.929238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f567bf-8aeb-4bc2-93ed-f5e36c9348d4 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:36.934482 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a875b-86b5-4fb5-a16f-64c2a1bd13b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:36.935438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a875b-86b5-4fb5-a16f-64c2a1bd13b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.935722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a875b-86b5-4fb5-a16f-64c2a1bd13b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.935995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a875b-86b5-4fb5-a16f-64c2a1bd13b2 admin admin] PATH_INFO: `/v3/projects/f87417e1032f44428a44848612c06297/users/a27725e89ee74ab39c275e9e3e790540/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.938020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a875b-86b5-4fb5-a16f-64c2a1bd13b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87417e1032f44428a44848612c06297, user_id=a27725e89ee74ab39c275e9e3e790540, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:36.940707 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a875b-86b5-4fb5-a16f-64c2a1bd13b2 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:36.952439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1266/2489] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:36 2022] PUT /identity/v3/projects/f87417e1032f44428a44848612c06297/users/a27725e89ee74ab39c275e9e3e790540/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:36.963636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f64a54-6f89-4abf-9bbe-fe297d0c458e None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:36.964426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f64a54-6f89-4abf-9bbe-fe297d0c458e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:36.964426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f64a54-6f89-4abf-9bbe-fe297d0c458e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:36.990563 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-82f64a54-6f89-4abf-9bbe-fe297d0c458e None None] MFA Rules not processed for user `a27725e89ee74ab39c275e9e3e790540`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:36.992331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-82f64a54-6f89-4abf-9bbe-fe297d0c458e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:37.060691 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1224/2490] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:36:37.074318 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a875b-86b5-4fb5-a16f-64c2a1bd13b2 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:37.078833 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-6c3a875b-86b5-4fb5-a16f-64c2a1bd13b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:37.160189 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a875b-86b5-4fb5-a16f-64c2a1bd13b2 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:37.164456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bebeb-2f5c-4f90-a3bf-692635f06831 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:37.165166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bebeb-2f5c-4f90-a3bf-692635f06831 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:37.165453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bebeb-2f5c-4f90-a3bf-692635f06831 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:37.165733 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bebeb-2f5c-4f90-a3bf-692635f06831 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:37.175472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9bebeb-2f5c-4f90-a3bf-692635f06831 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:37.177080 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9bebeb-2f5c-4f90-a3bf-692635f06831 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:37.188427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1267/2491] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:37.896462 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f64a54-6f89-4abf-9bbe-fe297d0c458e None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:37.969083 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f64a54-6f89-4abf-9bbe-fe297d0c458e None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:37.973919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8526d2-2661-48cc-a236-cd8e732c7282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:37.974783 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8526d2-2661-48cc-a236-cd8e732c7282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:37.975061 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8526d2-2661-48cc-a236-cd8e732c7282 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:37.975472 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8526d2-2661-48cc-a236-cd8e732c7282 admin admin] PATH_INFO: `/v3/users/6082d27a844b41a192d546200cf9521c` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:37.976645 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8526d2-2661-48cc-a236-cd8e732c7282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6082d27a844b41a192d546200cf9521c)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:37.978634 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8526d2-2661-48cc-a236-cd8e732c7282 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:38.024878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ae8526d2-2661-48cc-a236-cd8e732c7282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6082d27a844b41a192d546200cf9521c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.032617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ae8526d2-2661-48cc-a236-cd8e732c7282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6082d27a844b41a192d546200cf9521c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.033383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ae8526d2-2661-48cc-a236-cd8e732c7282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6082d27a844b41a192d546200cf9521c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.039799 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ae8526d2-2661-48cc-a236-cd8e732c7282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6082d27a844b41a192d546200cf9521c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.040511 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-ae8526d2-2661-48cc-a236-cd8e732c7282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6082d27a844b41a192d546200cf9521c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.057514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1225/2492] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:37 2022] DELETE /identity/v3/users/6082d27a844b41a192d546200cf9521c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:38.062024 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c9b37-2720-4286-a8f3-6b198545760b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:38.062483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c9b37-2720-4286-a8f3-6b198545760b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:38.062575 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c9b37-2720-4286-a8f3-6b198545760b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:38.080197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-849c9b37-2720-4286-a8f3-6b198545760b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:38.081375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-849c9b37-2720-4286-a8f3-6b198545760b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:38.137655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1268/2493] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:38.148845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8526d2-2661-48cc-a236-cd8e732c7282 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:38.151370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ae8526d2-2661-48cc-a236-cd8e732c7282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:38.207754 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8526d2-2661-48cc-a236-cd8e732c7282 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:38.211927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61497e0-808a-4395-bd6c-0fadead8e535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:38.214418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61497e0-808a-4395-bd6c-0fadead8e535 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:38.214418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61497e0-808a-4395-bd6c-0fadead8e535 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:38.214418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61497e0-808a-4395-bd6c-0fadead8e535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:38.222409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61497e0-808a-4395-bd6c-0fadead8e535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:38.223851 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61497e0-808a-4395-bd6c-0fadead8e535 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:38.233629 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1226/2494] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:36:38.519291 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c9b37-2720-4286-a8f3-6b198545760b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:38.520824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-849c9b37-2720-4286-a8f3-6b198545760b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:38.551908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c9b37-2720-4286-a8f3-6b198545760b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:38.556626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:38.557509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:38.557783 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:38.558051 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] PATH_INFO: `/v3/projects/3b5b0552533f45a197bbb74d4edc9ae9` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:38.559381 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b5b0552533f45a197bbb74d4edc9ae9)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:38.561272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:38.612271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b5b0552533f45a197bbb74d4edc9ae9', 'request_id': 'req-061f176b-1e8e-4b8a-95d8-01aecb1058a9'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.621264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b5b0552533f45a197bbb74d4edc9ae9', 'request_id': 'req-061f176b-1e8e-4b8a-95d8-01aecb1058a9'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.629137 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b5b0552533f45a197bbb74d4edc9ae9', 'request_id': 'req-061f176b-1e8e-4b8a-95d8-01aecb1058a9'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.641174 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] The token cache is being invalidate because project 3b5b0552533f45a197bbb74d4edc9ae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:38.641460 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.642097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.643794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1269/2495] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:38 2022] DELETE /identity/v3/projects/3b5b0552533f45a197bbb74d4edc9ae9 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:38.647895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61497e0-808a-4395-bd6c-0fadead8e535 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:38.657410 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61497e0-808a-4395-bd6c-0fadead8e535 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:38.719341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3428db2-51f8-4929-a4b4-89e03f88d546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:38.720283 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3428db2-51f8-4929-a4b4-89e03f88d546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:38.720571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3428db2-51f8-4929-a4b4-89e03f88d546 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:38.720845 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3428db2-51f8-4929-a4b4-89e03f88d546 admin admin] PATH_INFO: `/v3/users/a27725e89ee74ab39c275e9e3e790540` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:38.722047 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3428db2-51f8-4929-a4b4-89e03f88d546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a27725e89ee74ab39c275e9e3e790540)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:38.724169 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3428db2-51f8-4929-a4b4-89e03f88d546 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:38.785159 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c3428db2-51f8-4929-a4b4-89e03f88d546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a27725e89ee74ab39c275e9e3e790540'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.792451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c3428db2-51f8-4929-a4b4-89e03f88d546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a27725e89ee74ab39c275e9e3e790540'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.793114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c3428db2-51f8-4929-a4b4-89e03f88d546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a27725e89ee74ab39c275e9e3e790540'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.800628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c3428db2-51f8-4929-a4b4-89e03f88d546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a27725e89ee74ab39c275e9e3e790540'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.801251 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-c3428db2-51f8-4929-a4b4-89e03f88d546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a27725e89ee74ab39c275e9e3e790540'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:38.818423 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1227/2496] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:38 2022] DELETE /identity/v3/users/a27725e89ee74ab39c275e9e3e790540 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:39.101024 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:39.102454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:39.117395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f176b-1e8e-4b8a-95d8-01aecb1058a9 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:39.173172 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:39.174035 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:39.174320 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:39.174598 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43 admin admin] PATH_INFO: `/v3/projects/f87417e1032f44428a44848612c06297` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:39.175782 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f87417e1032f44428a44848612c06297)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:39.177668 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:39.222655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f87417e1032f44428a44848612c06297', 'request_id': 'req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:39.228000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f87417e1032f44428a44848612c06297', 'request_id': 'req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:39.234675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f87417e1032f44428a44848612c06297', 'request_id': 'req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:39.245264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43 admin admin] The token cache is being invalidate because project f87417e1032f44428a44848612c06297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:39.245550 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:39.246432 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-910fb6e6-73d6-4fe9-b7f8-977c9c1b0f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:39.248008 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1270/2497] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:39 2022] DELETE /identity/v3/projects/f87417e1032f44428a44848612c06297 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:39.252967 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae684afd-bbd3-4275-882d-a968faf85723 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:39.253451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae684afd-bbd3-4275-882d-a968faf85723 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:39.253550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae684afd-bbd3-4275-882d-a968faf85723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:39.271784 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ae684afd-bbd3-4275-882d-a968faf85723 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:39.273593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ae684afd-bbd3-4275-882d-a968faf85723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:39.328917 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1228/2498] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:39.337223 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4efa58-6fc7-40ab-b036-d51081aba263 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:39.337515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4efa58-6fc7-40ab-b036-d51081aba263 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:39.337830 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4efa58-6fc7-40ab-b036-d51081aba263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:39.356641 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-aa4efa58-6fc7-40ab-b036-d51081aba263 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:39.357821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-aa4efa58-6fc7-40ab-b036-d51081aba263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:39.366248 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1271/2499] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:39.370332 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae684afd-bbd3-4275-882d-a968faf85723 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:39.380240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae684afd-bbd3-4275-882d-a968faf85723 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:39.385464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6794f0-6bec-46c5-9669-7937523a676b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:39.386116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6794f0-6bec-46c5-9669-7937523a676b admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:39.386402 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6794f0-6bec-46c5-9669-7937523a676b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:39.386681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6794f0-6bec-46c5-9669-7937523a676b admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:39.387469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6794f0-6bec-46c5-9669-7937523a676b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:36:39.387780 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6794f0-6bec-46c5-9669-7937523a676b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:39.389364 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6794f0-6bec-46c5-9669-7937523a676b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:39.400934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1229/2500] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:36:39.405113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4efa58-6fc7-40ab-b036-d51081aba263 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:39.414021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4efa58-6fc7-40ab-b036-d51081aba263 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:39.418526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2a822-8337-4a08-98ee-56a94e1c9c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:39.419168 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2a822-8337-4a08-98ee-56a94e1c9c0c admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:39.419483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2a822-8337-4a08-98ee-56a94e1c9c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:39.419757 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2a822-8337-4a08-98ee-56a94e1c9c0c admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:39.420467 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2a822-8337-4a08-98ee-56a94e1c9c0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:39.422327 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2a822-8337-4a08-98ee-56a94e1c9c0c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:39.447875 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1272/2501] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:39 2022] POST /identity/v3/projects => generated 373 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:39.451316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6794f0-6bec-46c5-9669-7937523a676b admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:39.460907 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6794f0-6bec-46c5-9669-7937523a676b admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:39.515295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5c1cf-bdaf-40eb-b544-cdb8ca0faf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:39.516235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5c1cf-bdaf-40eb-b544-cdb8ca0faf3a admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:39.516503 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5c1cf-bdaf-40eb-b544-cdb8ca0faf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:39.516801 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5c1cf-bdaf-40eb-b544-cdb8ca0faf3a admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:39.517457 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5c1cf-bdaf-40eb-b544-cdb8ca0faf3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:39.519371 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5c1cf-bdaf-40eb-b544-cdb8ca0faf3a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:39.543005 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1230/2502] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:39 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:39.546937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2a822-8337-4a08-98ee-56a94e1c9c0c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:39.559624 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2a822-8337-4a08-98ee-56a94e1c9c0c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:39.570422 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7529e3-b655-412d-8439-77c2c53210da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:39.576190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7529e3-b655-412d-8439-77c2c53210da admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:39.576190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7529e3-b655-412d-8439-77c2c53210da admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:39.576651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7529e3-b655-412d-8439-77c2c53210da admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:39.577436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7529e3-b655-412d-8439-77c2c53210da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:39.579623 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7529e3-b655-412d-8439-77c2c53210da admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:39.593560 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1273/2503] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:36:39.598913 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5c1cf-bdaf-40eb-b544-cdb8ca0faf3a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:39.619302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5c1cf-bdaf-40eb-b544-cdb8ca0faf3a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:39.631051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b198b-b1b9-4e8b-8a02-b0bee15e4fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:39.632914 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b198b-b1b9-4e8b-8a02-b0bee15e4fdd admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:39.633102 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b198b-b1b9-4e8b-8a02-b0bee15e4fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:39.633381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b198b-b1b9-4e8b-8a02-b0bee15e4fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:39.634180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b198b-b1b9-4e8b-8a02-b0bee15e4fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:39.635994 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b198b-b1b9-4e8b-8a02-b0bee15e4fdd admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:39.651384 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1231/2504] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:39.653802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7529e3-b655-412d-8439-77c2c53210da admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:39.666315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7529e3-b655-412d-8439-77c2c53210da admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:39.671240 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de237fe7-d53f-433a-a796-d3ddf0707f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:39.672139 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de237fe7-d53f-433a-a796-d3ddf0707f52 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:39.672419 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de237fe7-d53f-433a-a796-d3ddf0707f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:39.672699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de237fe7-d53f-433a-a796-d3ddf0707f52 admin admin] PATH_INFO: `/v3/projects/0fdcad602a654b27bd50558d39c6c9dd/users/0fa089ce63f9423ea7f627cdcb9bb10a/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:39.689229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de237fe7-d53f-433a-a796-d3ddf0707f52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fdcad602a654b27bd50558d39c6c9dd, user_id=0fa089ce63f9423ea7f627cdcb9bb10a, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:39.692125 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de237fe7-d53f-433a-a796-d3ddf0707f52 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:39.707500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1274/2505] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:39 2022] PUT /identity/v3/projects/0fdcad602a654b27bd50558d39c6c9dd/users/0fa089ce63f9423ea7f627cdcb9bb10a/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:39.718477 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3bb3d-adad-4aa5-bbf8-4bc06e116bcd None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:39.718762 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3bb3d-adad-4aa5-bbf8-4bc06e116bcd None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:39.719042 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3bb3d-adad-4aa5-bbf8-4bc06e116bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:39.789763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-aaa3bb3d-adad-4aa5-bbf8-4bc06e116bcd None None] MFA Rules not processed for user `0fa089ce63f9423ea7f627cdcb9bb10a`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:39.792234 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-aaa3bb3d-adad-4aa5-bbf8-4bc06e116bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:39.802819 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de237fe7-d53f-433a-a796-d3ddf0707f52 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:39.806170 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-de237fe7-d53f-433a-a796-d3ddf0707f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:39.883540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1232/2506] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:39.908098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3bb3d-adad-4aa5-bbf8-4bc06e116bcd None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:39.909755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-aaa3bb3d-adad-4aa5-bbf8-4bc06e116bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:39.929803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de237fe7-d53f-433a-a796-d3ddf0707f52 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:39.934241 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaec28-72c2-4cbc-8656-d6cfbf9fbec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:39.935304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaec28-72c2-4cbc-8656-d6cfbf9fbec3 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:39.935685 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaec28-72c2-4cbc-8656-d6cfbf9fbec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:39.936082 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaec28-72c2-4cbc-8656-d6cfbf9fbec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:39.938268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-11aaec28-72c2-4cbc-8656-d6cfbf9fbec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:40.010526 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3bb3d-adad-4aa5-bbf8-4bc06e116bcd None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:40.014755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da5174-ffb4-4a1c-925f-1ff0c72370ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:40.015484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da5174-ffb4-4a1c-925f-1ff0c72370ee service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:40.015755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da5174-ffb4-4a1c-925f-1ff0c72370ee service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:40.016032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da5174-ffb4-4a1c-925f-1ff0c72370ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:40.017778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-81da5174-ffb4-4a1c-925f-1ff0c72370ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:40.023727 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aaec28-72c2-4cbc-8656-d6cfbf9fbec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:40.025156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aaec28-72c2-4cbc-8656-d6cfbf9fbec3 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:40.030302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da5174-ffb4-4a1c-925f-1ff0c72370ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:40.032089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da5174-ffb4-4a1c-925f-1ff0c72370ee service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:40.034956 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1275/2507] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:36:40.035635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaec28-72c2-4cbc-8656-d6cfbf9fbec3 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:40.036969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-11aaec28-72c2-4cbc-8656-d6cfbf9fbec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:40.043483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1233/2508] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:36:40.091670 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaec28-72c2-4cbc-8656-d6cfbf9fbec3 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:40.095352 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c44b3-16f6-4adc-8c1d-3de24727ecfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:40.096000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c44b3-16f6-4adc-8c1d-3de24727ecfc service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:40.096283 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c44b3-16f6-4adc-8c1d-3de24727ecfc service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:40.096558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c44b3-16f6-4adc-8c1d-3de24727ecfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:40.105307 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c44b3-16f6-4adc-8c1d-3de24727ecfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:40.106725 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c44b3-16f6-4adc-8c1d-3de24727ecfc service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:40.116259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1276/2509] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:40.135867 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da5174-ffb4-4a1c-925f-1ff0c72370ee service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:40.145669 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da5174-ffb4-4a1c-925f-1ff0c72370ee service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:40.152937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8d10c-01d2-4591-b130-485b08ab2b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:40.154155 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8d10c-01d2-4591-b130-485b08ab2b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:40.154771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8d10c-01d2-4591-b130-485b08ab2b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:40.155405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8d10c-01d2-4591-b130-485b08ab2b29 admin admin] PATH_INFO: `/v3/users/0fa089ce63f9423ea7f627cdcb9bb10a` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:40.156925 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8d10c-01d2-4591-b130-485b08ab2b29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fa089ce63f9423ea7f627cdcb9bb10a)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:40.158855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8d10c-01d2-4591-b130-485b08ab2b29 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:40.214176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f1b8d10c-01d2-4591-b130-485b08ab2b29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fa089ce63f9423ea7f627cdcb9bb10a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:40.221078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f1b8d10c-01d2-4591-b130-485b08ab2b29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fa089ce63f9423ea7f627cdcb9bb10a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:40.221787 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f1b8d10c-01d2-4591-b130-485b08ab2b29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fa089ce63f9423ea7f627cdcb9bb10a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:40.229270 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f1b8d10c-01d2-4591-b130-485b08ab2b29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fa089ce63f9423ea7f627cdcb9bb10a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:40.229953 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f1b8d10c-01d2-4591-b130-485b08ab2b29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fa089ce63f9423ea7f627cdcb9bb10a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:40.247765 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1234/2510] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:40 2022] DELETE /identity/v3/users/0fa089ce63f9423ea7f627cdcb9bb10a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:40.251991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e931b5a-4330-4ef9-bdb5-199131558d26 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:40.252331 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e931b5a-4330-4ef9-bdb5-199131558d26 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:40.252616 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e931b5a-4330-4ef9-bdb5-199131558d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:40.269342 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-7e931b5a-4330-4ef9-bdb5-199131558d26 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:40.270521 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7e931b5a-4330-4ef9-bdb5-199131558d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:40.326593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1277/2511] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:40.336814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8d10c-01d2-4591-b130-485b08ab2b29 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:40.338993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f1b8d10c-01d2-4591-b130-485b08ab2b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:40.397806 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8d10c-01d2-4591-b130-485b08ab2b29 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:40.401457 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c812c2a-4410-4eb5-b243-292aeae1f316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:40.402348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c812c2a-4410-4eb5-b243-292aeae1f316 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:40.402348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c812c2a-4410-4eb5-b243-292aeae1f316 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:40.402637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c812c2a-4410-4eb5-b243-292aeae1f316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:40.412157 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c812c2a-4410-4eb5-b243-292aeae1f316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:40.414158 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c812c2a-4410-4eb5-b243-292aeae1f316 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:40.423526 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1235/2512] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:40.687579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e931b5a-4330-4ef9-bdb5-199131558d26 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:40.689115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7e931b5a-4330-4ef9-bdb5-199131558d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:40.702136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e931b5a-4330-4ef9-bdb5-199131558d26 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:40.707876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e50283-e955-4162-afee-515cff45f78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:40.708877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e50283-e955-4162-afee-515cff45f78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:40.709260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e50283-e955-4162-afee-515cff45f78b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:40.709664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e50283-e955-4162-afee-515cff45f78b admin admin] PATH_INFO: `/v3/projects/0fdcad602a654b27bd50558d39c6c9dd` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:40.710950 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e50283-e955-4162-afee-515cff45f78b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fdcad602a654b27bd50558d39c6c9dd)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:40.713182 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e50283-e955-4162-afee-515cff45f78b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:40.760730 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e3e50283-e955-4162-afee-515cff45f78b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fdcad602a654b27bd50558d39c6c9dd', 'request_id': 'req-e3e50283-e955-4162-afee-515cff45f78b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:40.765875 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e3e50283-e955-4162-afee-515cff45f78b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fdcad602a654b27bd50558d39c6c9dd', 'request_id': 'req-e3e50283-e955-4162-afee-515cff45f78b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:40.773342 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e3e50283-e955-4162-afee-515cff45f78b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fdcad602a654b27bd50558d39c6c9dd', 'request_id': 'req-e3e50283-e955-4162-afee-515cff45f78b'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:40.785529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e3e50283-e955-4162-afee-515cff45f78b admin admin] The token cache is being invalidate because project 0fdcad602a654b27bd50558d39c6c9dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:40.785813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e3e50283-e955-4162-afee-515cff45f78b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:40.786470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-e3e50283-e955-4162-afee-515cff45f78b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:40.788611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1278/2513] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:40 2022] DELETE /identity/v3/projects/0fdcad602a654b27bd50558d39c6c9dd => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:40.793724 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae28b0d-bb2a-43fc-8c1c-72bdd0cec9de None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:40.794006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae28b0d-bb2a-43fc-8c1c-72bdd0cec9de None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:40.794285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae28b0d-bb2a-43fc-8c1c-72bdd0cec9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:40.813187 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-eae28b0d-bb2a-43fc-8c1c-72bdd0cec9de None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:40.814406 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-eae28b0d-bb2a-43fc-8c1c-72bdd0cec9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:40.876196 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1236/2514] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:36:40.885117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bb457-983a-4a1c-aca1-b80db5370822 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:40.885402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bb457-983a-4a1c-aca1-b80db5370822 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:40.885685 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bb457-983a-4a1c-aca1-b80db5370822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:40.904851 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-3d7bb457-983a-4a1c-aca1-b80db5370822 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:40.906059 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3d7bb457-983a-4a1c-aca1-b80db5370822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:40.914747 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1279/2515] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:40.919067 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae28b0d-bb2a-43fc-8c1c-72bdd0cec9de None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:40.928924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae28b0d-bb2a-43fc-8c1c-72bdd0cec9de None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:40.933654 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56f68b-ca4f-454f-8439-83bfcf8afc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:40.934261 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56f68b-ca4f-454f-8439-83bfcf8afc5c admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:40.934543 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56f68b-ca4f-454f-8439-83bfcf8afc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:40.934816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56f68b-ca4f-454f-8439-83bfcf8afc5c admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:40.935586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56f68b-ca4f-454f-8439-83bfcf8afc5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:36:40.935891 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56f68b-ca4f-454f-8439-83bfcf8afc5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:40.937493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56f68b-ca4f-454f-8439-83bfcf8afc5c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:40.950382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1237/2516] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:36:40.955109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bb457-983a-4a1c-aca1-b80db5370822 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:40.966439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bb457-983a-4a1c-aca1-b80db5370822 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:40.974820 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc72dee-1513-4047-8ba1-10e2d1db023e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:40.976188 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc72dee-1513-4047-8ba1-10e2d1db023e admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:40.976543 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc72dee-1513-4047-8ba1-10e2d1db023e admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:40.976889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc72dee-1513-4047-8ba1-10e2d1db023e admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:40.977673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc72dee-1513-4047-8ba1-10e2d1db023e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:40.979727 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc72dee-1513-4047-8ba1-10e2d1db023e admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:41.013848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1280/2517] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:40 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:36:41.014997 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56f68b-ca4f-454f-8439-83bfcf8afc5c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:41.027443 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56f68b-ca4f-454f-8439-83bfcf8afc5c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:41.096035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd4e8d-dbe1-4206-a2a2-5cff3ac818b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:41.096714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd4e8d-dbe1-4206-a2a2-5cff3ac818b9 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:41.096999 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd4e8d-dbe1-4206-a2a2-5cff3ac818b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:41.097274 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd4e8d-dbe1-4206-a2a2-5cff3ac818b9 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:41.097950 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd4e8d-dbe1-4206-a2a2-5cff3ac818b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:41.099892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd4e8d-dbe1-4206-a2a2-5cff3ac818b9 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:41.121596 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1238/2518] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:41 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:41.125386 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc72dee-1513-4047-8ba1-10e2d1db023e admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:41.138201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc72dee-1513-4047-8ba1-10e2d1db023e admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:41.144843 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd95ac-e337-4b2f-94d9-5da351f7169a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:41.145472 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd95ac-e337-4b2f-94d9-5da351f7169a admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:41.145750 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd95ac-e337-4b2f-94d9-5da351f7169a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:41.146022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd95ac-e337-4b2f-94d9-5da351f7169a admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:41.149880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd95ac-e337-4b2f-94d9-5da351f7169a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:41.151735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd95ac-e337-4b2f-94d9-5da351f7169a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:41.163934 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1281/2519] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:41.168631 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd4e8d-dbe1-4206-a2a2-5cff3ac818b9 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:41.178554 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd4e8d-dbe1-4206-a2a2-5cff3ac818b9 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:41.183394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c63473-0e36-4d8e-9e98-4c80f752b894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:41.184320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c63473-0e36-4d8e-9e98-4c80f752b894 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:41.184645 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c63473-0e36-4d8e-9e98-4c80f752b894 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:41.185024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c63473-0e36-4d8e-9e98-4c80f752b894 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:41.185865 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c63473-0e36-4d8e-9e98-4c80f752b894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:41.187575 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c63473-0e36-4d8e-9e98-4c80f752b894 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:41.199214 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1239/2520] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:41.201593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd95ac-e337-4b2f-94d9-5da351f7169a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:41.212305 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd95ac-e337-4b2f-94d9-5da351f7169a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:41.216961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9ae86-4dea-4890-b80a-64cd802ced93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:41.217913 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9ae86-4dea-4890-b80a-64cd802ced93 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:41.218258 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9ae86-4dea-4890-b80a-64cd802ced93 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:41.218596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9ae86-4dea-4890-b80a-64cd802ced93 admin admin] PATH_INFO: `/v3/projects/a5860bdd9b984d0d900c4874327738d0/users/8379643d645f40058fd5a7992d186676/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:41.234295 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9ae86-4dea-4890-b80a-64cd802ced93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5860bdd9b984d0d900c4874327738d0, user_id=8379643d645f40058fd5a7992d186676, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:41.237051 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9ae86-4dea-4890-b80a-64cd802ced93 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:41.252021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1282/2521] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:41 2022] PUT /identity/v3/projects/a5860bdd9b984d0d900c4874327738d0/users/8379643d645f40058fd5a7992d186676/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:41.262492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b55d4-02ef-4662-ad85-8fa7c54985e3 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:41.262779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b55d4-02ef-4662-ad85-8fa7c54985e3 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:41.263056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b55d4-02ef-4662-ad85-8fa7c54985e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:41.289561 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-379b55d4-02ef-4662-ad85-8fa7c54985e3 None None] MFA Rules not processed for user `8379643d645f40058fd5a7992d186676`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:41.290749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-379b55d4-02ef-4662-ad85-8fa7c54985e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:41.355458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1240/2522] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:41.359842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9ae86-4dea-4890-b80a-64cd802ced93 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:41.424168 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9ae86-4dea-4890-b80a-64cd802ced93 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:41.429331 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0acc6-54c1-45b9-87e2-99a4eb9b1486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:41.430101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0acc6-54c1-45b9-87e2-99a4eb9b1486 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:41.430603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0acc6-54c1-45b9-87e2-99a4eb9b1486 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:41.430944 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0acc6-54c1-45b9-87e2-99a4eb9b1486 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:41.431745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0acc6-54c1-45b9-87e2-99a4eb9b1486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:41.433691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0acc6-54c1-45b9-87e2-99a4eb9b1486 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:41.464568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1283/2523] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:41 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:36:41.468603 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b55d4-02ef-4662-ad85-8fa7c54985e3 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:41.479282 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b55d4-02ef-4662-ad85-8fa7c54985e3 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:41.538840 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cd34e-40e3-4afc-99fd-491233b8a185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:41.539836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cd34e-40e3-4afc-99fd-491233b8a185 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:41.540278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cd34e-40e3-4afc-99fd-491233b8a185 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:41.540749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cd34e-40e3-4afc-99fd-491233b8a185 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:41.541605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923cd34e-40e3-4afc-99fd-491233b8a185 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:41.543855 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923cd34e-40e3-4afc-99fd-491233b8a185 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:41.566643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1241/2524] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:41 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:36:41.571464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0acc6-54c1-45b9-87e2-99a4eb9b1486 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:41.585177 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0acc6-54c1-45b9-87e2-99a4eb9b1486 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:41.590067 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b114711-1d33-4836-866d-de6e063b2f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:41.591668 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b114711-1d33-4836-866d-de6e063b2f8c admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:41.592057 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b114711-1d33-4836-866d-de6e063b2f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:41.592403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b114711-1d33-4836-866d-de6e063b2f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:41.593191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b114711-1d33-4836-866d-de6e063b2f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:41.594859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b114711-1d33-4836-866d-de6e063b2f8c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:41.608380 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1284/2525] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:36:41.612483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cd34e-40e3-4afc-99fd-491233b8a185 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:41.623201 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cd34e-40e3-4afc-99fd-491233b8a185 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:41.628155 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82601b64-80d4-4edc-97c2-cd08ac4bc70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:41.629057 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82601b64-80d4-4edc-97c2-cd08ac4bc70f admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:41.629368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82601b64-80d4-4edc-97c2-cd08ac4bc70f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:41.629617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82601b64-80d4-4edc-97c2-cd08ac4bc70f admin admin] PATH_INFO: `/v3/projects/f5efb0be35164015b475a1df8b2a9c12/users/4b0f3b885d6541c8a039063907f40914/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:41.645379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82601b64-80d4-4edc-97c2-cd08ac4bc70f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5efb0be35164015b475a1df8b2a9c12, user_id=4b0f3b885d6541c8a039063907f40914, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:41.648714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82601b64-80d4-4edc-97c2-cd08ac4bc70f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:41.667014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1242/2526] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:41 2022] PUT /identity/v3/projects/f5efb0be35164015b475a1df8b2a9c12/users/4b0f3b885d6541c8a039063907f40914/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:41.668806 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b114711-1d33-4836-866d-de6e063b2f8c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:41.680090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b114711-1d33-4836-866d-de6e063b2f8c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:41.734862 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cca6b-33b3-4dc3-b492-8484f1fc91d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:41.735479 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cca6b-33b3-4dc3-b492-8484f1fc91d7 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:41.735769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cca6b-33b3-4dc3-b492-8484f1fc91d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:41.736044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cca6b-33b3-4dc3-b492-8484f1fc91d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:41.736758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cca6b-33b3-4dc3-b492-8484f1fc91d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:41.738378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cca6b-33b3-4dc3-b492-8484f1fc91d7 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:41.749831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1285/2527] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:41.754193 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82601b64-80d4-4edc-97c2-cd08ac4bc70f admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:41.765025 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82601b64-80d4-4edc-97c2-cd08ac4bc70f admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:41.769711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8ff70-3ba8-4db9-92b4-9db1f62f0b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:41.770678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8ff70-3ba8-4db9-92b4-9db1f62f0b22 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:41.771037 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8ff70-3ba8-4db9-92b4-9db1f62f0b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:41.771404 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8ff70-3ba8-4db9-92b4-9db1f62f0b22 admin admin] PATH_INFO: `/v3/projects/f5efb0be35164015b475a1df8b2a9c12/users/4b0f3b885d6541c8a039063907f40914/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:41.773434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8ff70-3ba8-4db9-92b4-9db1f62f0b22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5efb0be35164015b475a1df8b2a9c12, user_id=4b0f3b885d6541c8a039063907f40914, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:41.776188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8ff70-3ba8-4db9-92b4-9db1f62f0b22 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:41.786273 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1243/2528] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:41 2022] PUT /identity/v3/projects/f5efb0be35164015b475a1df8b2a9c12/users/4b0f3b885d6541c8a039063907f40914/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:41.797390 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f04d1-fabe-4f4b-9c4f-982d8212b973 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:41.797756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f04d1-fabe-4f4b-9c4f-982d8212b973 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:41.798140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f04d1-fabe-4f4b-9c4f-982d8212b973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:41.825115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-738f04d1-fabe-4f4b-9c4f-982d8212b973 None None] MFA Rules not processed for user `4b0f3b885d6541c8a039063907f40914`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:41.826381 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-738f04d1-fabe-4f4b-9c4f-982d8212b973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:41.896444 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1286/2529] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:41.907863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8ff70-3ba8-4db9-92b4-9db1f62f0b22 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:41.909604 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f1a8ff70-3ba8-4db9-92b4-9db1f62f0b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:41.970826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8ff70-3ba8-4db9-92b4-9db1f62f0b22 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:41.976178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c645b19-0707-4132-a644-059b959254da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:41.976931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c645b19-0707-4132-a644-059b959254da service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:41.977212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c645b19-0707-4132-a644-059b959254da service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:41.977489 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c645b19-0707-4132-a644-059b959254da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:42.043163 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c645b19-0707-4132-a644-059b959254da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:42.045133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c645b19-0707-4132-a644-059b959254da service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:42.057851 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1244/2530] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:45.724431 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f04d1-fabe-4f4b-9c4f-982d8212b973 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:45.726531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-738f04d1-fabe-4f4b-9c4f-982d8212b973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:45.791754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f04d1-fabe-4f4b-9c4f-982d8212b973 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:45.796370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad1840-e6c0-4aa2-9b47-9ba846f541f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:45.797314 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad1840-e6c0-4aa2-9b47-9ba846f541f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:45.797663 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad1840-e6c0-4aa2-9b47-9ba846f541f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:45.797942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad1840-e6c0-4aa2-9b47-9ba846f541f0 admin admin] PATH_INFO: `/v3/users/90fdc96d6e6649a498bb3a8659fcc030` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:45.799116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad1840-e6c0-4aa2-9b47-9ba846f541f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90fdc96d6e6649a498bb3a8659fcc030)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:45.801049 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad1840-e6c0-4aa2-9b47-9ba846f541f0 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:45.848586 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-58ad1840-e6c0-4aa2-9b47-9ba846f541f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90fdc96d6e6649a498bb3a8659fcc030'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:45.848912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-58ad1840-e6c0-4aa2-9b47-9ba846f541f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90fdc96d6e6649a498bb3a8659fcc030'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:45.858011 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-58ad1840-e6c0-4aa2-9b47-9ba846f541f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90fdc96d6e6649a498bb3a8659fcc030'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:45.867576 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-58ad1840-e6c0-4aa2-9b47-9ba846f541f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90fdc96d6e6649a498bb3a8659fcc030'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:45.868345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-58ad1840-e6c0-4aa2-9b47-9ba846f541f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90fdc96d6e6649a498bb3a8659fcc030'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:45.890318 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1287/2531] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:45 2022] DELETE /identity/v3/users/90fdc96d6e6649a498bb3a8659fcc030 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:46.076815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c645b19-0707-4132-a644-059b959254da service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:46.093466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c645b19-0707-4132-a644-059b959254da service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:46.163024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcb738-e2db-4798-87d3-bf2acc79e837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:46.163943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcb738-e2db-4798-87d3-bf2acc79e837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:46.164227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcb738-e2db-4798-87d3-bf2acc79e837 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:46.164507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcb738-e2db-4798-87d3-bf2acc79e837 admin admin] PATH_INFO: `/v3/projects/78039fb4f0bd454b8aea3db748343263` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:46.165701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fcb738-e2db-4798-87d3-bf2acc79e837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78039fb4f0bd454b8aea3db748343263)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:46.168187 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fcb738-e2db-4798-87d3-bf2acc79e837 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:46.249306 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-18fcb738-e2db-4798-87d3-bf2acc79e837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78039fb4f0bd454b8aea3db748343263', 'request_id': 'req-18fcb738-e2db-4798-87d3-bf2acc79e837'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:46.257016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-18fcb738-e2db-4798-87d3-bf2acc79e837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78039fb4f0bd454b8aea3db748343263', 'request_id': 'req-18fcb738-e2db-4798-87d3-bf2acc79e837'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:46.268114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-18fcb738-e2db-4798-87d3-bf2acc79e837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78039fb4f0bd454b8aea3db748343263', 'request_id': 'req-18fcb738-e2db-4798-87d3-bf2acc79e837'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:46.281686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-18fcb738-e2db-4798-87d3-bf2acc79e837 admin admin] The token cache is being invalidate because project 78039fb4f0bd454b8aea3db748343263 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:46.282109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-18fcb738-e2db-4798-87d3-bf2acc79e837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:46.282873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-18fcb738-e2db-4798-87d3-bf2acc79e837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:46.285703 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1245/2532] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:46 2022] DELETE /identity/v3/projects/78039fb4f0bd454b8aea3db748343263 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:46.290961 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb45ab-4d00-4c98-9c8c-df6194526b9e None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:46.291279 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb45ab-4d00-4c98-9c8c-df6194526b9e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:46.291565 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb45ab-4d00-4c98-9c8c-df6194526b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:46.310044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-fddb45ab-4d00-4c98-9c8c-df6194526b9e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:46.311600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-fddb45ab-4d00-4c98-9c8c-df6194526b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:46.384691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1288/2533] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:46.392609 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf391-b1c0-4fec-834e-edc8be5be1a2 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:46.392902 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf391-b1c0-4fec-834e-edc8be5be1a2 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:46.393157 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf391-b1c0-4fec-834e-edc8be5be1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:46.413172 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-babdf391-b1c0-4fec-834e-edc8be5be1a2 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:46.414368 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-babdf391-b1c0-4fec-834e-edc8be5be1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:46.422259 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1246/2534] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:46.426492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb45ab-4d00-4c98-9c8c-df6194526b9e None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:46.437233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb45ab-4d00-4c98-9c8c-df6194526b9e None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:46.444345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa20a4-c1ab-4680-9f85-b6498c92c631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:46.445289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa20a4-c1ab-4680-9f85-b6498c92c631 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:46.445526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa20a4-c1ab-4680-9f85-b6498c92c631 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:46.446004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa20a4-c1ab-4680-9f85-b6498c92c631 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:46.446762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa20a4-c1ab-4680-9f85-b6498c92c631 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:36:46.447187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa20a4-c1ab-4680-9f85-b6498c92c631 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:46.449057 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa20a4-c1ab-4680-9f85-b6498c92c631 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:46.465586 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1289/2535] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:36:46.466764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdf391-b1c0-4fec-834e-edc8be5be1a2 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:46.476542 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdf391-b1c0-4fec-834e-edc8be5be1a2 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:46.482473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690ac20-d7fd-485e-80c5-bc8f4afd48f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:46.482860 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690ac20-d7fd-485e-80c5-bc8f4afd48f8 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:46.483149 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690ac20-d7fd-485e-80c5-bc8f4afd48f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:46.483470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690ac20-d7fd-485e-80c5-bc8f4afd48f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:46.484155 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690ac20-d7fd-485e-80c5-bc8f4afd48f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:46.486070 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690ac20-d7fd-485e-80c5-bc8f4afd48f8 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:46.516668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1247/2536] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:46 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:46.520834 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa20a4-c1ab-4680-9f85-b6498c92c631 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:46.532824 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa20a4-c1ab-4680-9f85-b6498c92c631 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:46.589313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142c39d-1ad4-447f-9d58-866d1ac196e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:46.589992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142c39d-1ad4-447f-9d58-866d1ac196e4 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:46.590274 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142c39d-1ad4-447f-9d58-866d1ac196e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:46.590544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142c39d-1ad4-447f-9d58-866d1ac196e4 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:46.591425 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142c39d-1ad4-447f-9d58-866d1ac196e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:46.593465 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142c39d-1ad4-447f-9d58-866d1ac196e4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:46.617917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1290/2537] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:46 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:36:46.620777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690ac20-d7fd-485e-80c5-bc8f4afd48f8 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:46.634469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690ac20-d7fd-485e-80c5-bc8f4afd48f8 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:46.641676 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7522e79-c75b-49d7-9e26-49a454048a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:46.642444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7522e79-c75b-49d7-9e26-49a454048a85 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:46.642966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7522e79-c75b-49d7-9e26-49a454048a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:46.643451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7522e79-c75b-49d7-9e26-49a454048a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:46.644198 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7522e79-c75b-49d7-9e26-49a454048a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:46.645819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7522e79-c75b-49d7-9e26-49a454048a85 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:46.667409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1248/2538] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:46.671766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142c39d-1ad4-447f-9d58-866d1ac196e4 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:46.683776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142c39d-1ad4-447f-9d58-866d1ac196e4 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:46.688684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6263ab2-1bca-46f1-9b10-1cedfcef87c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:46.694183 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6263ab2-1bca-46f1-9b10-1cedfcef87c3 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:46.694479 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6263ab2-1bca-46f1-9b10-1cedfcef87c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:46.694763 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6263ab2-1bca-46f1-9b10-1cedfcef87c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:46.695555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6263ab2-1bca-46f1-9b10-1cedfcef87c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:46.697179 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6263ab2-1bca-46f1-9b10-1cedfcef87c3 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:46.707680 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1291/2539] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:46.712000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7522e79-c75b-49d7-9e26-49a454048a85 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:46.722488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7522e79-c75b-49d7-9e26-49a454048a85 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:46.728991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ef4e6-959a-4614-937f-24a9c6b66aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:46.730238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ef4e6-959a-4614-937f-24a9c6b66aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:46.730513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ef4e6-959a-4614-937f-24a9c6b66aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:46.730790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ef4e6-959a-4614-937f-24a9c6b66aa7 admin admin] PATH_INFO: `/v3/projects/dca809e484154187a0be3148ae99b42b/users/4da644234f224251852b5ade850c9680/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:46.751407 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ef4e6-959a-4614-937f-24a9c6b66aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dca809e484154187a0be3148ae99b42b, user_id=4da644234f224251852b5ade850c9680, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:46.754700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ef4e6-959a-4614-937f-24a9c6b66aa7 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:46.778663 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1249/2540] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:46 2022] PUT /identity/v3/projects/dca809e484154187a0be3148ae99b42b/users/4da644234f224251852b5ade850c9680/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:46.780489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cdd3d-a436-4be7-8549-30a5af75d766 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:46.780954 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cdd3d-a436-4be7-8549-30a5af75d766 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:46.781050 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cdd3d-a436-4be7-8549-30a5af75d766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:46.816511 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-fc3cdd3d-a436-4be7-8549-30a5af75d766 None None] MFA Rules not processed for user `4da644234f224251852b5ade850c9680`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:46.818185 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-fc3cdd3d-a436-4be7-8549-30a5af75d766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:46.887642 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1292/2541] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:46 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:46.892016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ef4e6-959a-4614-937f-24a9c6b66aa7 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:46.924675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cdd3d-a436-4be7-8549-30a5af75d766 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:46.926706 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-fc3cdd3d-a436-4be7-8549-30a5af75d766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:46.968181 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ef4e6-959a-4614-937f-24a9c6b66aa7 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:46.974370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2a76c-ae1c-41b1-80fe-7b24a2a5a38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:46.975628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2a76c-ae1c-41b1-80fe-7b24a2a5a38c admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:46.975912 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2a76c-ae1c-41b1-80fe-7b24a2a5a38c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:46.976183 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2a76c-ae1c-41b1-80fe-7b24a2a5a38c admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:46.976864 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2a76c-ae1c-41b1-80fe-7b24a2a5a38c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:46.978961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2a76c-ae1c-41b1-80fe-7b24a2a5a38c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:47.003664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cdd3d-a436-4be7-8549-30a5af75d766 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:47.006755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1250/2542] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:46 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:47.010320 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a4f1b-40e3-4ac3-bb45-1347a9ea796e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:47.010682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2a76c-ae1c-41b1-80fe-7b24a2a5a38c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:47.011222 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a4f1b-40e3-4ac3-bb45-1347a9ea796e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:47.011523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a4f1b-40e3-4ac3-bb45-1347a9ea796e admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:47.011853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a4f1b-40e3-4ac3-bb45-1347a9ea796e admin admin] PATH_INFO: `/v3/users/8379643d645f40058fd5a7992d186676` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:47.013114 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a4f1b-40e3-4ac3-bb45-1347a9ea796e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8379643d645f40058fd5a7992d186676)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:47.015133 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a4f1b-40e3-4ac3-bb45-1347a9ea796e admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:47.064619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de0a4f1b-40e3-4ac3-bb45-1347a9ea796e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8379643d645f40058fd5a7992d186676'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:47.065337 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de0a4f1b-40e3-4ac3-bb45-1347a9ea796e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8379643d645f40058fd5a7992d186676'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:47.073457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de0a4f1b-40e3-4ac3-bb45-1347a9ea796e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8379643d645f40058fd5a7992d186676'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:47.079304 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2a76c-ae1c-41b1-80fe-7b24a2a5a38c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:47.081827 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de0a4f1b-40e3-4ac3-bb45-1347a9ea796e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8379643d645f40058fd5a7992d186676'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:47.082797 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de0a4f1b-40e3-4ac3-bb45-1347a9ea796e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8379643d645f40058fd5a7992d186676'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:47.084838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c8f5c-3130-4781-a5b8-573d0c6bcffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:47.085519 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c8f5c-3130-4781-a5b8-573d0c6bcffe admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:47.085802 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c8f5c-3130-4781-a5b8-573d0c6bcffe admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:47.086222 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c8f5c-3130-4781-a5b8-573d0c6bcffe admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:47.087032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c8f5c-3130-4781-a5b8-573d0c6bcffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:47.088922 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c8f5c-3130-4781-a5b8-573d0c6bcffe admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:47.105256 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1293/2543] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:46 2022] DELETE /identity/v3/users/8379643d645f40058fd5a7992d186676 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:47.109684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181cdb3c-74be-444f-8933-c8434858615e None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:47.109972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181cdb3c-74be-444f-8933-c8434858615e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:47.110247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181cdb3c-74be-444f-8933-c8434858615e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:47.113584 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1251/2544] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:47 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:47.117431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c8f5c-3130-4781-a5b8-573d0c6bcffe admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:47.119350 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-618c8f5c-3130-4781-a5b8-573d0c6bcffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:47.130109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-181cdb3c-74be-444f-8933-c8434858615e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:47.131318 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-181cdb3c-74be-444f-8933-c8434858615e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:47.134950 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c8f5c-3130-4781-a5b8-573d0c6bcffe admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:47.202664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1294/2545] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:47.213902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181cdb3c-74be-444f-8933-c8434858615e None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:47.215364 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-181cdb3c-74be-444f-8933-c8434858615e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:47.227749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd96cc4-4471-412d-9882-5e2b0a174490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:47.228718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd96cc4-4471-412d-9882-5e2b0a174490 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:47.229194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd96cc4-4471-412d-9882-5e2b0a174490 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:47.229479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd96cc4-4471-412d-9882-5e2b0a174490 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:47.230269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd96cc4-4471-412d-9882-5e2b0a174490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:47.231917 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd96cc4-4471-412d-9882-5e2b0a174490 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:47.244380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1252/2546] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:47.248298 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd96cc4-4471-412d-9882-5e2b0a174490 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:47.261201 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd96cc4-4471-412d-9882-5e2b0a174490 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:47.268832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a392125-6313-406c-890e-84e2c9952625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:47.269757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a392125-6313-406c-890e-84e2c9952625 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:47.270044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a392125-6313-406c-890e-84e2c9952625 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:47.270327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a392125-6313-406c-890e-84e2c9952625 admin admin] PATH_INFO: `/v3/projects/f96f5cb2640545fb929e2b1ef641c4f3/users/e35afe7563b54838b03af01313ea6916/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:47.282186 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181cdb3c-74be-444f-8933-c8434858615e None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:47.287210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fd680-eccf-44e2-9849-02bf4412cbe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:47.287210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fd680-eccf-44e2-9849-02bf4412cbe5 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:47.287515 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fd680-eccf-44e2-9849-02bf4412cbe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:47.287611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fd680-eccf-44e2-9849-02bf4412cbe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:47.290540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a392125-6313-406c-890e-84e2c9952625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96f5cb2640545fb929e2b1ef641c4f3, user_id=e35afe7563b54838b03af01313ea6916, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:47.293699 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a392125-6313-406c-890e-84e2c9952625 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:47.298161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fd680-eccf-44e2-9849-02bf4412cbe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:47.299887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fd680-eccf-44e2-9849-02bf4412cbe5 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:47.311127 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1295/2547] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:36:47.314083 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1253/2548] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:47 2022] PUT /identity/v3/projects/f96f5cb2640545fb929e2b1ef641c4f3/users/e35afe7563b54838b03af01313ea6916/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:47.317873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fd680-eccf-44e2-9849-02bf4412cbe5 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:47.330717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fd680-eccf-44e2-9849-02bf4412cbe5 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:47.420479 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec973756-02bb-4e55-8000-1f07fdb32ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:47.421118 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec973756-02bb-4e55-8000-1f07fdb32ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:47.421406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec973756-02bb-4e55-8000-1f07fdb32ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:47.421697 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec973756-02bb-4e55-8000-1f07fdb32ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:47.422416 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec973756-02bb-4e55-8000-1f07fdb32ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:47.424208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec973756-02bb-4e55-8000-1f07fdb32ca9 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:47.435702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1296/2549] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:47.440143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a392125-6313-406c-890e-84e2c9952625 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:47.450636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a392125-6313-406c-890e-84e2c9952625 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:47.455992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a41245-a408-4ed6-bc82-1a79ba2e09ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:47.456697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a41245-a408-4ed6-bc82-1a79ba2e09ec admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:47.456985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a41245-a408-4ed6-bc82-1a79ba2e09ec admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:47.457309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a41245-a408-4ed6-bc82-1a79ba2e09ec admin admin] PATH_INFO: `/v3/projects/f96f5cb2640545fb929e2b1ef641c4f3/users/e35afe7563b54838b03af01313ea6916/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:47.460256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a41245-a408-4ed6-bc82-1a79ba2e09ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96f5cb2640545fb929e2b1ef641c4f3, user_id=e35afe7563b54838b03af01313ea6916, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:47.463411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a41245-a408-4ed6-bc82-1a79ba2e09ec admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:47.476665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1254/2550] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:47 2022] PUT /identity/v3/projects/f96f5cb2640545fb929e2b1ef641c4f3/users/e35afe7563b54838b03af01313ea6916/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:47.486696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4a8c2-4f3d-4bac-8c5e-82831638ef35 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:47.486978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4a8c2-4f3d-4bac-8c5e-82831638ef35 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:47.487286 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4a8c2-4f3d-4bac-8c5e-82831638ef35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:47.566231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-69b4a8c2-4f3d-4bac-8c5e-82831638ef35 None None] MFA Rules not processed for user `e35afe7563b54838b03af01313ea6916`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:47.571330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-69b4a8c2-4f3d-4bac-8c5e-82831638ef35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:47.717637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a41245-a408-4ed6-bc82-1a79ba2e09ec admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:47.720416 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-34a41245-a408-4ed6-bc82-1a79ba2e09ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:47.734569 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1297/2551] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:47.743414 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4a8c2-4f3d-4bac-8c5e-82831638ef35 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:47.746384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-69b4a8c2-4f3d-4bac-8c5e-82831638ef35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:47.819351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a41245-a408-4ed6-bc82-1a79ba2e09ec admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:47.824540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:47.825481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:47.825823 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:47.826158 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] PATH_INFO: `/v3/projects/a5860bdd9b984d0d900c4874327738d0` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:47.827816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5860bdd9b984d0d900c4874327738d0)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:47.829852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:47.837370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4a8c2-4f3d-4bac-8c5e-82831638ef35 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:47.842422 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4d2c4-878b-48fb-bbe5-931472accf2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:47.842879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d2c4-878b-48fb-bbe5-931472accf2a service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:47.843492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d2c4-878b-48fb-bbe5-931472accf2a service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:47.843611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d2c4-878b-48fb-bbe5-931472accf2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:47.854200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4d2c4-878b-48fb-bbe5-931472accf2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:47.855972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4d2c4-878b-48fb-bbe5-931472accf2a service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:47.879361 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1298/2552] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:36:47.892374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5860bdd9b984d0d900c4874327738d0', 'request_id': 'req-98f995c9-101b-4b26-82ae-36ded3f32970'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:47.903250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5860bdd9b984d0d900c4874327738d0', 'request_id': 'req-98f995c9-101b-4b26-82ae-36ded3f32970'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:47.915414 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5860bdd9b984d0d900c4874327738d0', 'request_id': 'req-98f995c9-101b-4b26-82ae-36ded3f32970'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:47.929267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] The token cache is being invalidate because project a5860bdd9b984d0d900c4874327738d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:47.929820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:47.930345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:47.933474 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1255/2553] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:47 2022] DELETE /identity/v3/projects/a5860bdd9b984d0d900c4874327738d0 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:47.937660 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4d2c4-878b-48fb-bbe5-931472accf2a service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:47.948762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4d2c4-878b-48fb-bbe5-931472accf2a service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:48.069481 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f32b3-5b37-41f2-aa69-c0dec1299c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:48.070364 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f32b3-5b37-41f2-aa69-c0dec1299c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:48.070651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f32b3-5b37-41f2-aa69-c0dec1299c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:48.070927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f32b3-5b37-41f2-aa69-c0dec1299c27 admin admin] PATH_INFO: `/v3/users/4b0f3b885d6541c8a039063907f40914` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:48.072176 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f32b3-5b37-41f2-aa69-c0dec1299c27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b0f3b885d6541c8a039063907f40914)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:48.074068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f32b3-5b37-41f2-aa69-c0dec1299c27 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:48.134822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-b64f32b3-5b37-41f2-aa69-c0dec1299c27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b0f3b885d6541c8a039063907f40914'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.135519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-b64f32b3-5b37-41f2-aa69-c0dec1299c27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b0f3b885d6541c8a039063907f40914'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.143941 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-b64f32b3-5b37-41f2-aa69-c0dec1299c27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b0f3b885d6541c8a039063907f40914'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.152058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-b64f32b3-5b37-41f2-aa69-c0dec1299c27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b0f3b885d6541c8a039063907f40914'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.152754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-b64f32b3-5b37-41f2-aa69-c0dec1299c27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b0f3b885d6541c8a039063907f40914'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.173676 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1299/2554] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:47 2022] DELETE /identity/v3/users/4b0f3b885d6541c8a039063907f40914 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:48.473225 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:48.474852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:48.489555 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f995c9-101b-4b26-82ae-36ded3f32970 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:48.524374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f32b3-5b37-41f2-aa69-c0dec1299c27 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:48.525832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b64f32b3-5b37-41f2-aa69-c0dec1299c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:48.569295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b98974-5588-4e9c-bb7f-6aee4a421307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:48.570500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b98974-5588-4e9c-bb7f-6aee4a421307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:48.570786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b98974-5588-4e9c-bb7f-6aee4a421307 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:48.571232 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b98974-5588-4e9c-bb7f-6aee4a421307 admin admin] PATH_INFO: `/v3/users/4da644234f224251852b5ade850c9680` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:48.572652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b98974-5588-4e9c-bb7f-6aee4a421307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4da644234f224251852b5ade850c9680)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:48.574622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b98974-5588-4e9c-bb7f-6aee4a421307 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:48.609621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f32b3-5b37-41f2-aa69-c0dec1299c27 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:48.616595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c677f-2326-4892-ad20-344aff94ab25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:48.617150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c677f-2326-4892-ad20-344aff94ab25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:48.617443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c677f-2326-4892-ad20-344aff94ab25 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:48.617699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c677f-2326-4892-ad20-344aff94ab25 admin admin] PATH_INFO: `/v3/projects/f5efb0be35164015b475a1df8b2a9c12` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:48.619340 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c677f-2326-4892-ad20-344aff94ab25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5efb0be35164015b475a1df8b2a9c12)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:48.621828 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c677f-2326-4892-ad20-344aff94ab25 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:48.625686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e0b98974-5588-4e9c-bb7f-6aee4a421307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4da644234f224251852b5ade850c9680'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.635637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e0b98974-5588-4e9c-bb7f-6aee4a421307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4da644234f224251852b5ade850c9680'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.636341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e0b98974-5588-4e9c-bb7f-6aee4a421307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4da644234f224251852b5ade850c9680'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.645254 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e0b98974-5588-4e9c-bb7f-6aee4a421307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4da644234f224251852b5ade850c9680'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.645942 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-e0b98974-5588-4e9c-bb7f-6aee4a421307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4da644234f224251852b5ade850c9680'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.668009 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1256/2555] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:48 2022] DELETE /identity/v3/users/4da644234f224251852b5ade850c9680 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:48.671822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-163c677f-2326-4892-ad20-344aff94ab25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5efb0be35164015b475a1df8b2a9c12', 'request_id': 'req-163c677f-2326-4892-ad20-344aff94ab25'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.672055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cadf02-00c9-42ee-9890-71b63c504cf2 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:48.672523 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cadf02-00c9-42ee-9890-71b63c504cf2 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:48.673155 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cadf02-00c9-42ee-9890-71b63c504cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:48.677389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-163c677f-2326-4892-ad20-344aff94ab25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5efb0be35164015b475a1df8b2a9c12', 'request_id': 'req-163c677f-2326-4892-ad20-344aff94ab25'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.685734 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-163c677f-2326-4892-ad20-344aff94ab25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5efb0be35164015b475a1df8b2a9c12', 'request_id': 'req-163c677f-2326-4892-ad20-344aff94ab25'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.693190 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-a1cadf02-00c9-42ee-9890-71b63c504cf2 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:48.694717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a1cadf02-00c9-42ee-9890-71b63c504cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:48.698150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-163c677f-2326-4892-ad20-344aff94ab25 admin admin] The token cache is being invalidate because project f5efb0be35164015b475a1df8b2a9c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:48.698436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-163c677f-2326-4892-ad20-344aff94ab25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.699088 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-163c677f-2326-4892-ad20-344aff94ab25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:48.700737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1300/2556] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:48 2022] DELETE /identity/v3/projects/f5efb0be35164015b475a1df8b2a9c12 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:48.705336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c2b26-e57d-446f-93ce-425ddf60c1a6 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:48.705629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c2b26-e57d-446f-93ce-425ddf60c1a6 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:48.705908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c2b26-e57d-446f-93ce-425ddf60c1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:48.725853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-855c2b26-e57d-446f-93ce-425ddf60c1a6 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:48.727049 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-855c2b26-e57d-446f-93ce-425ddf60c1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:48.764445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1257/2557] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:36:48.774893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cadf02-00c9-42ee-9890-71b63c504cf2 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:48.776297 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a1cadf02-00c9-42ee-9890-71b63c504cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:48.821602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1301/2558] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:36:48.830791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d9448-8ea3-4c27-b9fc-076be8a133f5 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:48.831130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d9448-8ea3-4c27-b9fc-076be8a133f5 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:48.832213 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d9448-8ea3-4c27-b9fc-076be8a133f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:48.852700 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-538d9448-8ea3-4c27-b9fc-076be8a133f5 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:48.853878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-538d9448-8ea3-4c27-b9fc-076be8a133f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:48.862323 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cadf02-00c9-42ee-9890-71b63c504cf2 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:48.868655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1302/2559] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:36:48.869008 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ac733-2c65-4524-9e9a-ac3cebd185f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:48.869828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ac733-2c65-4524-9e9a-ac3cebd185f3 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:48.870260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ac733-2c65-4524-9e9a-ac3cebd185f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:48.871015 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ac733-2c65-4524-9e9a-ac3cebd185f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:48.878445 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d9448-8ea3-4c27-b9fc-076be8a133f5 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:48.885967 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ac733-2c65-4524-9e9a-ac3cebd185f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:48.888536 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ac733-2c65-4524-9e9a-ac3cebd185f3 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:48.889577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d9448-8ea3-4c27-b9fc-076be8a133f5 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:48.894767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59038d92-3edb-468c-ac54-f5874fbdfd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:48.895494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59038d92-3edb-468c-ac54-f5874fbdfd36 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:48.895883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59038d92-3edb-468c-ac54-f5874fbdfd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:48.896158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59038d92-3edb-468c-ac54-f5874fbdfd36 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:48.896960 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59038d92-3edb-468c-ac54-f5874fbdfd36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:36:48.897297 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59038d92-3edb-468c-ac54-f5874fbdfd36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:48.899125 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59038d92-3edb-468c-ac54-f5874fbdfd36 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:48.904012 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1258/2560] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:36:48.915349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1303/2561] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:36:48.917195 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ac733-2c65-4524-9e9a-ac3cebd185f3 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:48.931389 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ac733-2c65-4524-9e9a-ac3cebd185f3 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:48.936376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5215b0-9327-4bba-bf05-16f71af1ea15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:48.937341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5215b0-9327-4bba-bf05-16f71af1ea15 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:48.937813 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5215b0-9327-4bba-bf05-16f71af1ea15 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:48.938271 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5215b0-9327-4bba-bf05-16f71af1ea15 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:48.939405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5215b0-9327-4bba-bf05-16f71af1ea15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:48.941842 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5215b0-9327-4bba-bf05-16f71af1ea15 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:48.980167 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1259/2562] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:48 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:48.984359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59038d92-3edb-468c-ac54-f5874fbdfd36 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:48.996975 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59038d92-3edb-468c-ac54-f5874fbdfd36 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:49.066926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36d45d-2a0c-4e2c-ac82-19a43c50dfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:49.067896 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36d45d-2a0c-4e2c-ac82-19a43c50dfec admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:49.068958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36d45d-2a0c-4e2c-ac82-19a43c50dfec admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:49.069287 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36d45d-2a0c-4e2c-ac82-19a43c50dfec admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:49.069988 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36d45d-2a0c-4e2c-ac82-19a43c50dfec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:49.072110 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36d45d-2a0c-4e2c-ac82-19a43c50dfec admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:49.103114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5215b0-9327-4bba-bf05-16f71af1ea15 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:49.111334 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1304/2563] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:48 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:49.121240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5215b0-9327-4bba-bf05-16f71af1ea15 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:49.129147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd99168-ef7d-4c31-a97f-f9a0fa4b7ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:49.129978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd99168-ef7d-4c31-a97f-f9a0fa4b7ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:49.130267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd99168-ef7d-4c31-a97f-f9a0fa4b7ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:49.130544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd99168-ef7d-4c31-a97f-f9a0fa4b7ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:49.131965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd99168-ef7d-4c31-a97f-f9a0fa4b7ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:49.133823 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd99168-ef7d-4c31-a97f-f9a0fa4b7ed8 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:49.153901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1260/2564] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:49.154664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36d45d-2a0c-4e2c-ac82-19a43c50dfec admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:49.165025 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36d45d-2a0c-4e2c-ac82-19a43c50dfec admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:49.170733 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d53301-ea71-4f4b-ab33-afdf19f904bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:49.171669 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d53301-ea71-4f4b-ab33-afdf19f904bd admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:49.172144 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d53301-ea71-4f4b-ab33-afdf19f904bd admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:49.172654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d53301-ea71-4f4b-ab33-afdf19f904bd admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:49.173631 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d53301-ea71-4f4b-ab33-afdf19f904bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:49.175539 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d53301-ea71-4f4b-ab33-afdf19f904bd admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:49.187049 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1305/2565] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:49.191139 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd99168-ef7d-4c31-a97f-f9a0fa4b7ed8 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:49.204617 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd99168-ef7d-4c31-a97f-f9a0fa4b7ed8 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:49.211082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e230319-26c7-4fb6-9d1d-8ba5cba99a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:49.212458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e230319-26c7-4fb6-9d1d-8ba5cba99a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:49.212802 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e230319-26c7-4fb6-9d1d-8ba5cba99a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:49.213388 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e230319-26c7-4fb6-9d1d-8ba5cba99a40 admin admin] PATH_INFO: `/v3/projects/84f9c806f7d54963bd886eae2ab22c38/users/d1934a2e2edb4b8a93baac501c9c5c3b/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:49.249241 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e230319-26c7-4fb6-9d1d-8ba5cba99a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f9c806f7d54963bd886eae2ab22c38, user_id=d1934a2e2edb4b8a93baac501c9c5c3b, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:49.252141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e230319-26c7-4fb6-9d1d-8ba5cba99a40 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:49.269001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1261/2566] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:49 2022] PUT /identity/v3/projects/84f9c806f7d54963bd886eae2ab22c38/users/d1934a2e2edb4b8a93baac501c9c5c3b/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:49.274202 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41378c-d8fd-43ec-9d71-06481b9d52c2 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:49.274492 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41378c-d8fd-43ec-9d71-06481b9d52c2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:49.274869 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41378c-d8fd-43ec-9d71-06481b9d52c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:49.293471 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-6f41378c-d8fd-43ec-9d71-06481b9d52c2 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:49.294694 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-6f41378c-d8fd-43ec-9d71-06481b9d52c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:49.370176 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1306/2567] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:49.379927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e230319-26c7-4fb6-9d1d-8ba5cba99a40 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:49.382247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7e230319-26c7-4fb6-9d1d-8ba5cba99a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:49.437695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e230319-26c7-4fb6-9d1d-8ba5cba99a40 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:49.441253 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a3c34-3d34-4166-b7a1-80022338dcee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:49.441894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a3c34-3d34-4166-b7a1-80022338dcee service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:49.442172 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a3c34-3d34-4166-b7a1-80022338dcee service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:49.442450 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a3c34-3d34-4166-b7a1-80022338dcee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:49.452355 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a3c34-3d34-4166-b7a1-80022338dcee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:49.453761 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a3c34-3d34-4166-b7a1-80022338dcee service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:49.463093 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1262/2568] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:49.518904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41378c-d8fd-43ec-9d71-06481b9d52c2 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:49.521132 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-6f41378c-d8fd-43ec-9d71-06481b9d52c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:49.537594 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41378c-d8fd-43ec-9d71-06481b9d52c2 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:49.551920 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:49.553901 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:49.554058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:49.554328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] PATH_INFO: `/v3/projects/dca809e484154187a0be3148ae99b42b` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:49.555844 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dca809e484154187a0be3148ae99b42b)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:49.558035 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:49.616432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a3c34-3d34-4166-b7a1-80022338dcee service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:49.616959 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dca809e484154187a0be3148ae99b42b', 'request_id': 'req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.619588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ed8a3c34-3d34-4166-b7a1-80022338dcee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:49.624055 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dca809e484154187a0be3148ae99b42b', 'request_id': 'req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.634068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dca809e484154187a0be3148ae99b42b', 'request_id': 'req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.638331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a3c34-3d34-4166-b7a1-80022338dcee service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:49.649338 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] The token cache is being invalidate because project dca809e484154187a0be3148ae99b42b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:49.649714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.650404 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.652313 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1307/2569] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:49 2022] DELETE /identity/v3/projects/dca809e484154187a0be3148ae99b42b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:49.656185 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:49.657860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:49.720197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec9501-11f9-49b5-9fb7-256fc733e931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:49.721473 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec9501-11f9-49b5-9fb7-256fc733e931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:49.721839 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec9501-11f9-49b5-9fb7-256fc733e931 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:49.722228 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec9501-11f9-49b5-9fb7-256fc733e931 admin admin] PATH_INFO: `/v3/users/96dcd82937834f89889c43a1fefc1bff` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:49.723837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec9501-11f9-49b5-9fb7-256fc733e931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96dcd82937834f89889c43a1fefc1bff)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:49.726497 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec9501-11f9-49b5-9fb7-256fc733e931 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:49.733305 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc8c3f-7515-44f2-bf12-1637dc0c33c0 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:49.738187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460c2c6-c621-4a25-b7c0-4e54678905c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:49.739069 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460c2c6-c621-4a25-b7c0-4e54678905c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:49.739378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460c2c6-c621-4a25-b7c0-4e54678905c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:49.739696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460c2c6-c621-4a25-b7c0-4e54678905c4 admin admin] PATH_INFO: `/v3/users/e35afe7563b54838b03af01313ea6916` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:49.740971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460c2c6-c621-4a25-b7c0-4e54678905c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35afe7563b54838b03af01313ea6916)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:49.742861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460c2c6-c621-4a25-b7c0-4e54678905c4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:49.785671 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-51ec9501-11f9-49b5-9fb7-256fc733e931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96dcd82937834f89889c43a1fefc1bff'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.792706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-51ec9501-11f9-49b5-9fb7-256fc733e931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96dcd82937834f89889c43a1fefc1bff'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.793397 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-51ec9501-11f9-49b5-9fb7-256fc733e931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96dcd82937834f89889c43a1fefc1bff'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.801620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-51ec9501-11f9-49b5-9fb7-256fc733e931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96dcd82937834f89889c43a1fefc1bff'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.802379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-51ec9501-11f9-49b5-9fb7-256fc733e931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96dcd82937834f89889c43a1fefc1bff'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.804994 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2460c2c6-c621-4a25-b7c0-4e54678905c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35afe7563b54838b03af01313ea6916'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.805767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2460c2c6-c621-4a25-b7c0-4e54678905c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35afe7563b54838b03af01313ea6916'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.817203 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2460c2c6-c621-4a25-b7c0-4e54678905c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35afe7563b54838b03af01313ea6916'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.830147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2460c2c6-c621-4a25-b7c0-4e54678905c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35afe7563b54838b03af01313ea6916'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.831802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2460c2c6-c621-4a25-b7c0-4e54678905c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35afe7563b54838b03af01313ea6916'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:49.845759 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1263/2570] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:49 2022] DELETE /identity/v3/users/96dcd82937834f89889c43a1fefc1bff => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:49.855048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1308/2571] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:49 2022] DELETE /identity/v3/users/e35afe7563b54838b03af01313ea6916 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:50.168974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec9501-11f9-49b5-9fb7-256fc733e931 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:50.170595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-51ec9501-11f9-49b5-9fb7-256fc733e931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:50.246518 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec9501-11f9-49b5-9fb7-256fc733e931 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:50.307921 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:50.308796 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:50.309097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:50.309369 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] PATH_INFO: `/v3/projects/9c6e0a23a3414fca84248553e4f85005` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:50.310620 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c6e0a23a3414fca84248553e4f85005)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:50.312600 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:50.360592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c6e0a23a3414fca84248553e4f85005', 'request_id': 'req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.365960 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c6e0a23a3414fca84248553e4f85005', 'request_id': 'req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.373724 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c6e0a23a3414fca84248553e4f85005', 'request_id': 'req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.386147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] The token cache is being invalidate because project 9c6e0a23a3414fca84248553e4f85005 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:50.386434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.387081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.388670 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1264/2572] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:50 2022] DELETE /identity/v3/projects/9c6e0a23a3414fca84248553e4f85005 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:50.392968 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460c2c6-c621-4a25-b7c0-4e54678905c4 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:50.394974 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2460c2c6-c621-4a25-b7c0-4e54678905c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:50.409280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460c2c6-c621-4a25-b7c0-4e54678905c4 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:50.476314 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c20a27-fada-451d-89db-b7996453c3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:50.477304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c20a27-fada-451d-89db-b7996453c3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:50.478212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c20a27-fada-451d-89db-b7996453c3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:50.478603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c20a27-fada-451d-89db-b7996453c3e3 admin admin] PATH_INFO: `/v3/users/1be77b4735634f718f55bc6de7fd28c5` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:50.480278 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c20a27-fada-451d-89db-b7996453c3e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1be77b4735634f718f55bc6de7fd28c5)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:50.484130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c20a27-fada-451d-89db-b7996453c3e3 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:50.555641 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-97c20a27-fada-451d-89db-b7996453c3e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1be77b4735634f718f55bc6de7fd28c5'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.557059 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-97c20a27-fada-451d-89db-b7996453c3e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1be77b4735634f718f55bc6de7fd28c5'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.575775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-97c20a27-fada-451d-89db-b7996453c3e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1be77b4735634f718f55bc6de7fd28c5'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.583040 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:50.585167 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:50.588507 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-97c20a27-fada-451d-89db-b7996453c3e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1be77b4735634f718f55bc6de7fd28c5'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.589435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-97c20a27-fada-451d-89db-b7996453c3e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1be77b4735634f718f55bc6de7fd28c5'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.611303 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1309/2573] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:50 2022] DELETE /identity/v3/users/1be77b4735634f718f55bc6de7fd28c5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:50.611531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148f171-90ae-4ac8-9c4e-e58ae3463fcc admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:50.675093 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:50.676988 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:50.677478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:50.678476 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] PATH_INFO: `/v3/projects/f96f5cb2640545fb929e2b1ef641c4f3` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:50.680082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96f5cb2640545fb929e2b1ef641c4f3)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:50.682230 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:50.746897 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96f5cb2640545fb929e2b1ef641c4f3', 'request_id': 'req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.756982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96f5cb2640545fb929e2b1ef641c4f3', 'request_id': 'req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.767294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96f5cb2640545fb929e2b1ef641c4f3', 'request_id': 'req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.784250 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] The token cache is being invalidate because project f96f5cb2640545fb929e2b1ef641c4f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:50.785127 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.788099 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:50.790980 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1265/2574] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:50 2022] DELETE /identity/v3/projects/f96f5cb2640545fb929e2b1ef641c4f3 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:50.796619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dc9fa-b334-483c-9390-e140425743c1 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:50.797162 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dc9fa-b334-483c-9390-e140425743c1 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:50.797509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dc9fa-b334-483c-9390-e140425743c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:50.837959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:50.841300 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:50.845906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-d35dc9fa-b334-483c-9390-e140425743c1 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:50.848402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-d35dc9fa-b334-483c-9390-e140425743c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:50.955664 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b22ab-034d-4905-9cbb-4ef03e1aa92c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:50.962260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239fd35-6d73-43b9-a086-97cdb70b6fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:50.963726 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239fd35-6d73-43b9-a086-97cdb70b6fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:50.964116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239fd35-6d73-43b9-a086-97cdb70b6fec admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:50.964487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239fd35-6d73-43b9-a086-97cdb70b6fec admin admin] PATH_INFO: `/v3/projects/62d7aa11427b4370a7e450a99ca61594` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:50.966487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239fd35-6d73-43b9-a086-97cdb70b6fec admin admin] RBAC: Authorizing `identity:delete_project(project_id=62d7aa11427b4370a7e450a99ca61594)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:50.968469 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239fd35-6d73-43b9-a086-97cdb70b6fec admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:51.010934 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1310/2575] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:36:51.012377 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e862ce-5a5c-411c-831b-96ebc4c49f04 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:51.012736 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e862ce-5a5c-411c-831b-96ebc4c49f04 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:51.013081 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e862ce-5a5c-411c-831b-96ebc4c49f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:51.035042 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b1e862ce-5a5c-411c-831b-96ebc4c49f04 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:51.036442 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b1e862ce-5a5c-411c-831b-96ebc4c49f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:51.055514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6239fd35-6d73-43b9-a086-97cdb70b6fec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62d7aa11427b4370a7e450a99ca61594', 'request_id': 'req-6239fd35-6d73-43b9-a086-97cdb70b6fec'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:51.064305 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6239fd35-6d73-43b9-a086-97cdb70b6fec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62d7aa11427b4370a7e450a99ca61594', 'request_id': 'req-6239fd35-6d73-43b9-a086-97cdb70b6fec'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:51.078454 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6239fd35-6d73-43b9-a086-97cdb70b6fec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62d7aa11427b4370a7e450a99ca61594', 'request_id': 'req-6239fd35-6d73-43b9-a086-97cdb70b6fec'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:51.099087 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6239fd35-6d73-43b9-a086-97cdb70b6fec admin admin] The token cache is being invalidate because project 62d7aa11427b4370a7e450a99ca61594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:51.099324 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6239fd35-6d73-43b9-a086-97cdb70b6fec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:51.100509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-6239fd35-6d73-43b9-a086-97cdb70b6fec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:51.119207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1266/2576] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:50 2022] DELETE /identity/v3/projects/62d7aa11427b4370a7e450a99ca61594 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:51.167112 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239fd35-6d73-43b9-a086-97cdb70b6fec admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:51.167374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1311/2577] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:51.184381 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239fd35-6d73-43b9-a086-97cdb70b6fec admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:51.194955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c740d27-aecd-4158-bb6a-62a7c7ee6bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:51.195687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c740d27-aecd-4158-bb6a-62a7c7ee6bae admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:51.195974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c740d27-aecd-4158-bb6a-62a7c7ee6bae admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:51.198443 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c740d27-aecd-4158-bb6a-62a7c7ee6bae admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:51.199426 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c740d27-aecd-4158-bb6a-62a7c7ee6bae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:36:51.200154 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c740d27-aecd-4158-bb6a-62a7c7ee6bae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:51.201843 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c740d27-aecd-4158-bb6a-62a7c7ee6bae admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:51.226505 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1267/2578] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:36:51.228033 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e862ce-5a5c-411c-831b-96ebc4c49f04 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:51.242152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e862ce-5a5c-411c-831b-96ebc4c49f04 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:51.251554 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81654e-bdbf-4939-8a6a-7b3656b37bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:51.253157 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81654e-bdbf-4939-8a6a-7b3656b37bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:51.253664 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81654e-bdbf-4939-8a6a-7b3656b37bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:51.254108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81654e-bdbf-4939-8a6a-7b3656b37bf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:51.256435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a81654e-bdbf-4939-8a6a-7b3656b37bf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:51.258875 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a81654e-bdbf-4939-8a6a-7b3656b37bf8 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:51.297509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1312/2579] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:51 2022] POST /identity/v3/projects => generated 387 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:51.301685 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c740d27-aecd-4158-bb6a-62a7c7ee6bae admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:51.312986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c740d27-aecd-4158-bb6a-62a7c7ee6bae admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:51.398411 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782ee4-4daa-4424-92f0-1509939648f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:51.399208 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782ee4-4daa-4424-92f0-1509939648f6 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:51.399951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782ee4-4daa-4424-92f0-1509939648f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:51.399951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782ee4-4daa-4424-92f0-1509939648f6 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:51.400474 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20782ee4-4daa-4424-92f0-1509939648f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:51.402420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20782ee4-4daa-4424-92f0-1509939648f6 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:51.427342 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1268/2580] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:51 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:51.430900 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81654e-bdbf-4939-8a6a-7b3656b37bf8 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:51.449068 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81654e-bdbf-4939-8a6a-7b3656b37bf8 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:51.454190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e9f8d-e770-42b8-9750-3d67c17e396c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:51.454987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e9f8d-e770-42b8-9750-3d67c17e396c admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:51.455365 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e9f8d-e770-42b8-9750-3d67c17e396c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:51.455707 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e9f8d-e770-42b8-9750-3d67c17e396c admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:51.456505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e9f8d-e770-42b8-9750-3d67c17e396c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:51.458214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e9f8d-e770-42b8-9750-3d67c17e396c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:51.470270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1313/2581] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:51.474434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782ee4-4daa-4424-92f0-1509939648f6 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:51.485269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782ee4-4daa-4424-92f0-1509939648f6 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:51.489903 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2cdaa-43bc-4027-a26e-a17818a4ae10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:51.490627 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2cdaa-43bc-4027-a26e-a17818a4ae10 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:51.490913 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2cdaa-43bc-4027-a26e-a17818a4ae10 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:51.491318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2cdaa-43bc-4027-a26e-a17818a4ae10 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:51.491935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2cdaa-43bc-4027-a26e-a17818a4ae10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:51.493575 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2cdaa-43bc-4027-a26e-a17818a4ae10 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:51.507075 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1269/2582] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:36:51.508398 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e9f8d-e770-42b8-9750-3d67c17e396c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:51.518776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e9f8d-e770-42b8-9750-3d67c17e396c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:51.524841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4f867-1b68-48b4-aec5-623b06529d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:51.526055 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4f867-1b68-48b4-aec5-623b06529d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:51.526387 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4f867-1b68-48b4-aec5-623b06529d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:51.526726 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4f867-1b68-48b4-aec5-623b06529d02 admin admin] PATH_INFO: `/v3/projects/7ed826d49b3c43c9bf1b5d3f4161cbbe/users/2c90bfb7f2b14015a14bc088869b97f6/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:51.549661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e4f867-1b68-48b4-aec5-623b06529d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed826d49b3c43c9bf1b5d3f4161cbbe, user_id=2c90bfb7f2b14015a14bc088869b97f6, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:51.552347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e4f867-1b68-48b4-aec5-623b06529d02 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:51.567887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1314/2583] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:51 2022] PUT /identity/v3/projects/7ed826d49b3c43c9bf1b5d3f4161cbbe/users/2c90bfb7f2b14015a14bc088869b97f6/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:51.579879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1a25f-edca-45b9-a66c-630180356b8c None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:51.580709 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1a25f-edca-45b9-a66c-630180356b8c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:51.581002 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1a25f-edca-45b9-a66c-630180356b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:51.629678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-05d1a25f-edca-45b9-a66c-630180356b8c None None] MFA Rules not processed for user `2c90bfb7f2b14015a14bc088869b97f6`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:51.631024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-05d1a25f-edca-45b9-a66c-630180356b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:51.710841 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1270/2584] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:51.714882 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4f867-1b68-48b4-aec5-623b06529d02 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:51.784997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4f867-1b68-48b4-aec5-623b06529d02 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:51.790631 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2eafec-ad1d-4eb9-8823-1f6e7b9c7968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:51.791630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2eafec-ad1d-4eb9-8823-1f6e7b9c7968 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:51.791777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2eafec-ad1d-4eb9-8823-1f6e7b9c7968 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:51.792011 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2eafec-ad1d-4eb9-8823-1f6e7b9c7968 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:51.792704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2eafec-ad1d-4eb9-8823-1f6e7b9c7968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:51.794617 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2eafec-ad1d-4eb9-8823-1f6e7b9c7968 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:51.823689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1315/2585] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:51 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:51.824978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1a25f-edca-45b9-a66c-630180356b8c None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:51.836239 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1a25f-edca-45b9-a66c-630180356b8c None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:51.891267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf0b2f-8899-482c-a269-0e9983e26c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:51.892224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf0b2f-8899-482c-a269-0e9983e26c8a admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:51.892608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf0b2f-8899-482c-a269-0e9983e26c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:51.892892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf0b2f-8899-482c-a269-0e9983e26c8a admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:51.893579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf0b2f-8899-482c-a269-0e9983e26c8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:51.895523 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf0b2f-8899-482c-a269-0e9983e26c8a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:51.918895 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1271/2586] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:51 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:36:51.920074 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2eafec-ad1d-4eb9-8823-1f6e7b9c7968 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:51.930558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2eafec-ad1d-4eb9-8823-1f6e7b9c7968 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:51.935451 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c6e2-915c-4a54-8a8c-df0acea7099f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:51.936191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c6e2-915c-4a54-8a8c-df0acea7099f admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:51.936475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c6e2-915c-4a54-8a8c-df0acea7099f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:51.936758 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c6e2-915c-4a54-8a8c-df0acea7099f admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:51.937498 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4c6e2-915c-4a54-8a8c-df0acea7099f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:51.939138 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4c6e2-915c-4a54-8a8c-df0acea7099f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:51.961460 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1316/2587] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:36:51.963995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf0b2f-8899-482c-a269-0e9983e26c8a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:51.974266 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf0b2f-8899-482c-a269-0e9983e26c8a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:51.979512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dba24d-541c-486a-9fec-94f440bcf2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:51.980405 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dba24d-541c-486a-9fec-94f440bcf2d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:51.980686 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dba24d-541c-486a-9fec-94f440bcf2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:51.980967 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dba24d-541c-486a-9fec-94f440bcf2d0 admin admin] PATH_INFO: `/v3/projects/7d6d0a5258884981a7ee35ccac65cdbc/users/3c09e89e2b4c469096789a187bb8d270/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:52.003893 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dba24d-541c-486a-9fec-94f440bcf2d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d6d0a5258884981a7ee35ccac65cdbc, user_id=3c09e89e2b4c469096789a187bb8d270, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:52.010493 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dba24d-541c-486a-9fec-94f440bcf2d0 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:52.033946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1272/2588] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:51 2022] PUT /identity/v3/projects/7d6d0a5258884981a7ee35ccac65cdbc/users/3c09e89e2b4c469096789a187bb8d270/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:52.034935 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c6e2-915c-4a54-8a8c-df0acea7099f admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:52.054224 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c6e2-915c-4a54-8a8c-df0acea7099f admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:52.142494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704dfa2-7314-4672-b743-8423aefec2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:52.143210 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704dfa2-7314-4672-b743-8423aefec2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:52.143519 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704dfa2-7314-4672-b743-8423aefec2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:52.143800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704dfa2-7314-4672-b743-8423aefec2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:52.144972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704dfa2-7314-4672-b743-8423aefec2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:52.147722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704dfa2-7314-4672-b743-8423aefec2e9 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:52.163848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1317/2589] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:36:52.168131 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dba24d-541c-486a-9fec-94f440bcf2d0 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:52.181778 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dba24d-541c-486a-9fec-94f440bcf2d0 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:52.186938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d1fae-7dff-4f55-8c4c-1353eb235fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:52.188168 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d1fae-7dff-4f55-8c4c-1353eb235fb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:52.188627 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d1fae-7dff-4f55-8c4c-1353eb235fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:52.189204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d1fae-7dff-4f55-8c4c-1353eb235fb5 admin admin] PATH_INFO: `/v3/projects/7d6d0a5258884981a7ee35ccac65cdbc/users/3c09e89e2b4c469096789a187bb8d270/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:52.191836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d1fae-7dff-4f55-8c4c-1353eb235fb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d6d0a5258884981a7ee35ccac65cdbc, user_id=3c09e89e2b4c469096789a187bb8d270, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:52.194714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d1fae-7dff-4f55-8c4c-1353eb235fb5 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:52.214195 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1273/2590] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:52 2022] PUT /identity/v3/projects/7d6d0a5258884981a7ee35ccac65cdbc/users/3c09e89e2b4c469096789a187bb8d270/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:52.225345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea021e6-c908-4ad9-b7b2-5f54019cbbb0 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:52.226002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea021e6-c908-4ad9-b7b2-5f54019cbbb0 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:52.226473 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea021e6-c908-4ad9-b7b2-5f54019cbbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:52.277793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-bea021e6-c908-4ad9-b7b2-5f54019cbbb0 None None] MFA Rules not processed for user `3c09e89e2b4c469096789a187bb8d270`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:52.279347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-bea021e6-c908-4ad9-b7b2-5f54019cbbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:52.387516 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1318/2591] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:52.399472 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d1fae-7dff-4f55-8c4c-1353eb235fb5 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:52.401789 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e64d1fae-7dff-4f55-8c4c-1353eb235fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:52.488544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d1fae-7dff-4f55-8c4c-1353eb235fb5 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:52.495135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48228e-3f68-4a11-be1f-f3d99ecccb24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:52.496714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48228e-3f68-4a11-be1f-f3d99ecccb24 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:52.496997 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48228e-3f68-4a11-be1f-f3d99ecccb24 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:52.497275 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48228e-3f68-4a11-be1f-f3d99ecccb24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:52.511474 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48228e-3f68-4a11-be1f-f3d99ecccb24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:52.513072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48228e-3f68-4a11-be1f-f3d99ecccb24 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:52.526330 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1274/2592] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:52.599390 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea021e6-c908-4ad9-b7b2-5f54019cbbb0 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:52.681968 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea021e6-c908-4ad9-b7b2-5f54019cbbb0 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:52.691625 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45ae99-8216-4a21-93ed-70dbed5ed3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:52.693352 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45ae99-8216-4a21-93ed-70dbed5ed3da admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:52.693809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45ae99-8216-4a21-93ed-70dbed5ed3da admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:52.694413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45ae99-8216-4a21-93ed-70dbed5ed3da admin admin] PATH_INFO: `/v3/users/2c90bfb7f2b14015a14bc088869b97f6` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:52.696250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45ae99-8216-4a21-93ed-70dbed5ed3da admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c90bfb7f2b14015a14bc088869b97f6)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:52.698945 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45ae99-8216-4a21-93ed-70dbed5ed3da admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:52.762336 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de45ae99-8216-4a21-93ed-70dbed5ed3da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c90bfb7f2b14015a14bc088869b97f6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:52.763351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de45ae99-8216-4a21-93ed-70dbed5ed3da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c90bfb7f2b14015a14bc088869b97f6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:52.774428 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de45ae99-8216-4a21-93ed-70dbed5ed3da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c90bfb7f2b14015a14bc088869b97f6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:52.784854 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de45ae99-8216-4a21-93ed-70dbed5ed3da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c90bfb7f2b14015a14bc088869b97f6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:52.785580 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de45ae99-8216-4a21-93ed-70dbed5ed3da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c90bfb7f2b14015a14bc088869b97f6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:52.815362 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1319/2593] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:52 2022] DELETE /identity/v3/users/2c90bfb7f2b14015a14bc088869b97f6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:52.817004 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baae8f-466e-443a-b376-f3933c7ac208 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:52.817740 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baae8f-466e-443a-b376-f3933c7ac208 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:52.818048 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baae8f-466e-443a-b376-f3933c7ac208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:52.840162 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f9baae8f-466e-443a-b376-f3933c7ac208 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:52.841451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f9baae8f-466e-443a-b376-f3933c7ac208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:52.919217 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1275/2594] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:52.928320 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45ae99-8216-4a21-93ed-70dbed5ed3da admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:52.929893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-de45ae99-8216-4a21-93ed-70dbed5ed3da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:53.024411 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45ae99-8216-4a21-93ed-70dbed5ed3da admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:53.028508 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e21fa-f889-489e-a06c-5fbe7c2f2a73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:53.029218 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e21fa-f889-489e-a06c-5fbe7c2f2a73 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:53.029502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e21fa-f889-489e-a06c-5fbe7c2f2a73 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:53.029856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e21fa-f889-489e-a06c-5fbe7c2f2a73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:53.040611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e21fa-f889-489e-a06c-5fbe7c2f2a73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:53.042209 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e21fa-f889-489e-a06c-5fbe7c2f2a73 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:53.054252 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1320/2595] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:53.430379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9baae8f-466e-443a-b376-f3933c7ac208 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:53.431836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f9baae8f-466e-443a-b376-f3933c7ac208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:53.464524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9baae8f-466e-443a-b376-f3933c7ac208 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:53.475484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:53.476316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:53.476648 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:53.476879 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] PATH_INFO: `/v3/projects/7ed826d49b3c43c9bf1b5d3f4161cbbe` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:53.478085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ed826d49b3c43c9bf1b5d3f4161cbbe)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:53.480017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:53.580746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ed826d49b3c43c9bf1b5d3f4161cbbe', 'request_id': 'req-350d7d41-9f24-4561-b0bf-5960cee7ea0c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:53.591981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ed826d49b3c43c9bf1b5d3f4161cbbe', 'request_id': 'req-350d7d41-9f24-4561-b0bf-5960cee7ea0c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:53.610869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ed826d49b3c43c9bf1b5d3f4161cbbe', 'request_id': 'req-350d7d41-9f24-4561-b0bf-5960cee7ea0c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:53.636900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] The token cache is being invalidate because project 7ed826d49b3c43c9bf1b5d3f4161cbbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:53.637123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:53.637842 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:53.642464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1276/2596] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:53 2022] DELETE /identity/v3/projects/7ed826d49b3c43c9bf1b5d3f4161cbbe => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:53.643713 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e21fa-f889-489e-a06c-5fbe7c2f2a73 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:53.656445 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e21fa-f889-489e-a06c-5fbe7c2f2a73 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:53.747439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e39b50-2226-4db5-9e7b-c0b05dc0f9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:53.750272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e39b50-2226-4db5-9e7b-c0b05dc0f9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:53.750710 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e39b50-2226-4db5-9e7b-c0b05dc0f9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:53.751109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e39b50-2226-4db5-9e7b-c0b05dc0f9a4 admin admin] PATH_INFO: `/v3/users/3c09e89e2b4c469096789a187bb8d270` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:53.752783 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e39b50-2226-4db5-9e7b-c0b05dc0f9a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c09e89e2b4c469096789a187bb8d270)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:53.755326 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e39b50-2226-4db5-9e7b-c0b05dc0f9a4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:53.824368 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f4e39b50-2226-4db5-9e7b-c0b05dc0f9a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c09e89e2b4c469096789a187bb8d270'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:53.825369 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f4e39b50-2226-4db5-9e7b-c0b05dc0f9a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c09e89e2b4c469096789a187bb8d270'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:53.835276 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f4e39b50-2226-4db5-9e7b-c0b05dc0f9a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c09e89e2b4c469096789a187bb8d270'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:53.843988 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f4e39b50-2226-4db5-9e7b-c0b05dc0f9a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c09e89e2b4c469096789a187bb8d270'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:53.844566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-f4e39b50-2226-4db5-9e7b-c0b05dc0f9a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c09e89e2b4c469096789a187bb8d270'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:53.867106 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1321/2597] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:36:53 2022] DELETE /identity/v3/users/3c09e89e2b4c469096789a187bb8d270 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:54.292231 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:54.299234 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:54.321964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d7d41-9f24-4561-b0bf-5960cee7ea0c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:54.418400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4cf22-460a-41c0-8740-21e5c1209e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:54.419383 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4cf22-460a-41c0-8740-21e5c1209e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:54.419668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4cf22-460a-41c0-8740-21e5c1209e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:54.420409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4cf22-460a-41c0-8740-21e5c1209e8a admin admin] PATH_INFO: `/v3/projects/7d6d0a5258884981a7ee35ccac65cdbc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:54.421219 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4cf22-460a-41c0-8740-21e5c1209e8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d6d0a5258884981a7ee35ccac65cdbc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:54.423311 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4cf22-460a-41c0-8740-21e5c1209e8a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:54.482782 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37afb8d-556a-4376-bdba-f352b325c6e4 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:54.483405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37afb8d-556a-4376-bdba-f352b325c6e4 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:54.483405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37afb8d-556a-4376-bdba-f352b325c6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:54.494372 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-61d4cf22-460a-41c0-8740-21e5c1209e8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d6d0a5258884981a7ee35ccac65cdbc', 'request_id': 'req-61d4cf22-460a-41c0-8740-21e5c1209e8a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:54.502948 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-61d4cf22-460a-41c0-8740-21e5c1209e8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d6d0a5258884981a7ee35ccac65cdbc', 'request_id': 'req-61d4cf22-460a-41c0-8740-21e5c1209e8a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:54.514623 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-61d4cf22-460a-41c0-8740-21e5c1209e8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d6d0a5258884981a7ee35ccac65cdbc', 'request_id': 'req-61d4cf22-460a-41c0-8740-21e5c1209e8a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:54.514795 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-f37afb8d-556a-4376-bdba-f352b325c6e4 None None] MFA Rules not processed for user `d1934a2e2edb4b8a93baac501c9c5c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:54.516244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f37afb8d-556a-4376-bdba-f352b325c6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:54.529898 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-61d4cf22-460a-41c0-8740-21e5c1209e8a admin admin] The token cache is being invalidate because project 7d6d0a5258884981a7ee35ccac65cdbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:36:54.530363 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-61d4cf22-460a-41c0-8740-21e5c1209e8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:54.531032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-61d4cf22-460a-41c0-8740-21e5c1209e8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:36:54.533277 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1277/2598] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:36:54 2022] DELETE /identity/v3/projects/7d6d0a5258884981a7ee35ccac65cdbc => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:54.538134 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72fb5a-64e7-4bd7-b9c5-8c7a8acb2b1c None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:54.538479 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72fb5a-64e7-4bd7-b9c5-8c7a8acb2b1c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:54.538763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72fb5a-64e7-4bd7-b9c5-8c7a8acb2b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:54.574931 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-1f72fb5a-64e7-4bd7-b9c5-8c7a8acb2b1c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:54.576705 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1f72fb5a-64e7-4bd7-b9c5-8c7a8acb2b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:54.614456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1322/2599] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:54.623915 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37afb8d-556a-4376-bdba-f352b325c6e4 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:54.626120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f37afb8d-556a-4376-bdba-f352b325c6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:54.689849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1278/2600] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:36:54.703700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b97a6-5e9c-45a3-a4ac-3797f80e38d4 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:54.703991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b97a6-5e9c-45a3-a4ac-3797f80e38d4 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:54.704269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b97a6-5e9c-45a3-a4ac-3797f80e38d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:54.722577 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37afb8d-556a-4376-bdba-f352b325c6e4 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:54.730269 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c5bc3-44b8-4112-82b9-823a12efd9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:54.730996 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-d02b97a6-5e9c-45a3-a4ac-3797f80e38d4 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:54.731551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c5bc3-44b8-4112-82b9-823a12efd9ef admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:54.732010 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c5bc3-44b8-4112-82b9-823a12efd9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:54.732443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c5bc3-44b8-4112-82b9-823a12efd9ef admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:54.732906 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-d02b97a6-5e9c-45a3-a4ac-3797f80e38d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:54.733712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c5bc3-44b8-4112-82b9-823a12efd9ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:54.736609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c5bc3-44b8-4112-82b9-823a12efd9ef admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:54.746522 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1279/2601] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:36:54.750318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b97a6-5e9c-45a3-a4ac-3797f80e38d4 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:54.760871 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b97a6-5e9c-45a3-a4ac-3797f80e38d4 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:54.765949 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a744829-d283-4596-90c6-fe00e9fa10f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:54.766624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a744829-d283-4596-90c6-fe00e9fa10f1 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:54.766909 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a744829-d283-4596-90c6-fe00e9fa10f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:54.767256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a744829-d283-4596-90c6-fe00e9fa10f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:54.767978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a744829-d283-4596-90c6-fe00e9fa10f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:36:54.768294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a744829-d283-4596-90c6-fe00e9fa10f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:54.769892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a744829-d283-4596-90c6-fe00e9fa10f1 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:54.771927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1323/2602] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:54 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:36:54.775736 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c5bc3-44b8-4112-82b9-823a12efd9ef admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:54.787724 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1280/2603] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:36:54.791798 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a744829-d283-4596-90c6-fe00e9fa10f1 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:54.801911 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a744829-d283-4596-90c6-fe00e9fa10f1 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:54.851972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c5bc3-44b8-4112-82b9-823a12efd9ef admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:54.857867 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101619e-2b70-406e-8bf5-7e47da313b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:54.858607 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101619e-2b70-406e-8bf5-7e47da313b53 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:54.859276 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101619e-2b70-406e-8bf5-7e47da313b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:54.859575 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101619e-2b70-406e-8bf5-7e47da313b53 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:54.860484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101619e-2b70-406e-8bf5-7e47da313b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:54.862587 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101619e-2b70-406e-8bf5-7e47da313b53 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:54.887215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6604e4-8aaa-474c-9fcc-918433829507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:54.887215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6604e4-8aaa-474c-9fcc-918433829507 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:54.887215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6604e4-8aaa-474c-9fcc-918433829507 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:54.887215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6604e4-8aaa-474c-9fcc-918433829507 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:54.887732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6604e4-8aaa-474c-9fcc-918433829507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:54.887732 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6604e4-8aaa-474c-9fcc-918433829507 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:54.894318 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1324/2604] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:54 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:36:54.899090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101619e-2b70-406e-8bf5-7e47da313b53 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:54.909243 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101619e-2b70-406e-8bf5-7e47da313b53 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:54.914531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c5b82-08b0-468d-bb55-2a865e7f08a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:54.915346 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c5b82-08b0-468d-bb55-2a865e7f08a5 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:54.915707 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c5b82-08b0-468d-bb55-2a865e7f08a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:54.916046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c5b82-08b0-468d-bb55-2a865e7f08a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:54.916763 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c5b82-08b0-468d-bb55-2a865e7f08a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:54.918713 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c5b82-08b0-468d-bb55-2a865e7f08a5 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:54.930587 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1281/2605] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:54 2022] POST /identity/v3/projects => generated 391 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:36:54.930759 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1325/2606] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:54.931966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6604e4-8aaa-474c-9fcc-918433829507 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:54.935122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c5b82-08b0-468d-bb55-2a865e7f08a5 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:54.945980 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c5b82-08b0-468d-bb55-2a865e7f08a5 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:55.017629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada6e26-fc59-4a8a-8811-2df6a8b24469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:55.018740 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada6e26-fc59-4a8a-8811-2df6a8b24469 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:55.019031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada6e26-fc59-4a8a-8811-2df6a8b24469 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:55.019363 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada6e26-fc59-4a8a-8811-2df6a8b24469 admin admin] PATH_INFO: `/v3/projects/227044f3d2f94cea9da8413a52f95829/users/7cacb076763149cfab7464b1fbbe365d/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:55.026963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6604e4-8aaa-474c-9fcc-918433829507 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:55.032018 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1392505-dd81-438f-89d7-ea1145bfc8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:55.032894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1392505-dd81-438f-89d7-ea1145bfc8b3 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:55.033245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1392505-dd81-438f-89d7-ea1145bfc8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:55.033695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1392505-dd81-438f-89d7-ea1145bfc8b3 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:55.034334 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1392505-dd81-438f-89d7-ea1145bfc8b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:55.035892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada6e26-fc59-4a8a-8811-2df6a8b24469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=227044f3d2f94cea9da8413a52f95829, user_id=7cacb076763149cfab7464b1fbbe365d, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:55.036258 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1392505-dd81-438f-89d7-ea1145bfc8b3 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:55.039232 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada6e26-fc59-4a8a-8811-2df6a8b24469 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:55.058755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1326/2607] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:54 2022] PUT /identity/v3/projects/227044f3d2f94cea9da8413a52f95829/users/7cacb076763149cfab7464b1fbbe365d/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:55.060459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada6e26-fc59-4a8a-8811-2df6a8b24469 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:55.062186 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1282/2608] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:54 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:55.066113 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1392505-dd81-438f-89d7-ea1145bfc8b3 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:55.076791 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1392505-dd81-438f-89d7-ea1145bfc8b3 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:55.131483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada6e26-fc59-4a8a-8811-2df6a8b24469 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:55.137077 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de8342-f115-4701-b5b2-315421e19cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:55.137760 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de8342-f115-4701-b5b2-315421e19cda admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:55.138180 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de8342-f115-4701-b5b2-315421e19cda admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:55.138469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de8342-f115-4701-b5b2-315421e19cda admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:55.139409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de8342-f115-4701-b5b2-315421e19cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:55.141165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de8342-f115-4701-b5b2-315421e19cda admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:55.148388 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0844e4-0415-4e94-97f7-f278d1e886ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:55.149394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0844e4-0415-4e94-97f7-f278d1e886ef admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:55.149683 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0844e4-0415-4e94-97f7-f278d1e886ef admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:55.149956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0844e4-0415-4e94-97f7-f278d1e886ef admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:55.150689 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0844e4-0415-4e94-97f7-f278d1e886ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:55.152328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1327/2609] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:55.155351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0844e4-0415-4e94-97f7-f278d1e886ef admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:55.156229 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de8342-f115-4701-b5b2-315421e19cda admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:55.167358 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de8342-f115-4701-b5b2-315421e19cda admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:55.167661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1283/2610] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:36:55.171285 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0844e4-0415-4e94-97f7-f278d1e886ef admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:55.172458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c64174-0927-4341-ab54-81c361261194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:55.173351 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c64174-0927-4341-ab54-81c361261194 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:55.173629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c64174-0927-4341-ab54-81c361261194 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:55.173902 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c64174-0927-4341-ab54-81c361261194 admin admin] PATH_INFO: `/v3/projects/227044f3d2f94cea9da8413a52f95829/users/7cacb076763149cfab7464b1fbbe365d/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:55.176060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c64174-0927-4341-ab54-81c361261194 admin admin] RBAC: Authorizing `identity:create_grant(project_id=227044f3d2f94cea9da8413a52f95829, user_id=7cacb076763149cfab7464b1fbbe365d, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:55.178962 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c64174-0927-4341-ab54-81c361261194 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:55.183071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0844e4-0415-4e94-97f7-f278d1e886ef admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:55.190523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1328/2611] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:55 2022] PUT /identity/v3/projects/227044f3d2f94cea9da8413a52f95829/users/7cacb076763149cfab7464b1fbbe365d/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:36:55.196044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ffecd-89c6-4fee-9fc2-baefe69fff41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:55.197490 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ffecd-89c6-4fee-9fc2-baefe69fff41 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:55.198081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ffecd-89c6-4fee-9fc2-baefe69fff41 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:55.198524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ffecd-89c6-4fee-9fc2-baefe69fff41 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:55.199454 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ffecd-89c6-4fee-9fc2-baefe69fff41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:55.201641 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ffecd-89c6-4fee-9fc2-baefe69fff41 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:55.216604 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1284/2612] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:36:55.220877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c64174-0927-4341-ab54-81c361261194 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:55.308772 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c64174-0927-4341-ab54-81c361261194 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:55.313747 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0110b-6015-40fc-94be-9d1a1d0a3c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:55.314651 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0110b-6015-40fc-94be-9d1a1d0a3c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:55.315120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0110b-6015-40fc-94be-9d1a1d0a3c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:55.315507 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0110b-6015-40fc-94be-9d1a1d0a3c10 admin admin] PATH_INFO: `/v3/projects/cf022dc7daa24b4b8cbeae7ee9aec6a1/users/37a796c239c84047a3d6ea77f478ed7b/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:55.330989 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0110b-6015-40fc-94be-9d1a1d0a3c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf022dc7daa24b4b8cbeae7ee9aec6a1, user_id=37a796c239c84047a3d6ea77f478ed7b, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:55.334372 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0110b-6015-40fc-94be-9d1a1d0a3c10 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:55.349102 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1329/2613] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:55 2022] PUT /identity/v3/projects/cf022dc7daa24b4b8cbeae7ee9aec6a1/users/37a796c239c84047a3d6ea77f478ed7b/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:55.360907 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fb580-5869-4ceb-b336-73aebbb74463 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:55.361451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fb580-5869-4ceb-b336-73aebbb74463 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:55.361451 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fb580-5869-4ceb-b336-73aebbb74463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:55.390774 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-c81fb580-5869-4ceb-b336-73aebbb74463 None None] MFA Rules not processed for user `37a796c239c84047a3d6ea77f478ed7b`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:55.391946 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c81fb580-5869-4ceb-b336-73aebbb74463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:55.482526 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1285/2614] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:55.486937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0110b-6015-40fc-94be-9d1a1d0a3c10 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:55.589315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0110b-6015-40fc-94be-9d1a1d0a3c10 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:55.596023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2debf3-491e-434c-90eb-a3864e78cf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:55.596709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2debf3-491e-434c-90eb-a3864e78cf9c admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:55.597167 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2debf3-491e-434c-90eb-a3864e78cf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:55.597456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2debf3-491e-434c-90eb-a3864e78cf9c admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:55.598150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2debf3-491e-434c-90eb-a3864e78cf9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:55.600757 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2debf3-491e-434c-90eb-a3864e78cf9c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:55.641606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1330/2615] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:36:55 2022] POST /identity/v3/projects => generated 393 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:36:55.645633 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fb580-5869-4ceb-b336-73aebbb74463 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:55.659475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fb580-5869-4ceb-b336-73aebbb74463 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:55.728500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef372d3e-3e80-494b-993b-5212e845fb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:55.729341 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef372d3e-3e80-494b-993b-5212e845fb3a admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:55.730354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef372d3e-3e80-494b-993b-5212e845fb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:55.730354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef372d3e-3e80-494b-993b-5212e845fb3a admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:55.730919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef372d3e-3e80-494b-993b-5212e845fb3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:55.735976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef372d3e-3e80-494b-993b-5212e845fb3a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:55.772133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1286/2616] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:36:55 2022] POST /identity/v3/users => generated 303 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:36:55.775840 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2debf3-491e-434c-90eb-a3864e78cf9c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:55.787582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2debf3-491e-434c-90eb-a3864e78cf9c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:55.793316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f2e35-da86-4510-998c-fc5b3da36bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:55.794264 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f2e35-da86-4510-998c-fc5b3da36bbe admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:55.794568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f2e35-da86-4510-998c-fc5b3da36bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:55.794863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f2e35-da86-4510-998c-fc5b3da36bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:55.795706 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f2e35-da86-4510-998c-fc5b3da36bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:55.801362 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f2e35-da86-4510-998c-fc5b3da36bbe admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:55.814644 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1331/2617] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:36:55.818348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef372d3e-3e80-494b-993b-5212e845fb3a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:55.831868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef372d3e-3e80-494b-993b-5212e845fb3a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:55.837938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d815ab6-0b36-4f6f-94c8-ef62836a16f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:55.838906 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d815ab6-0b36-4f6f-94c8-ef62836a16f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:55.839445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d815ab6-0b36-4f6f-94c8-ef62836a16f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:55.840901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d815ab6-0b36-4f6f-94c8-ef62836a16f3 admin admin] PATH_INFO: `/v3/projects/17fc95390f2e4d68ac41d16c06deaee2/users/4fcbac00e1cc4609987d843bd35858b0/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:55.861672 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d815ab6-0b36-4f6f-94c8-ef62836a16f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17fc95390f2e4d68ac41d16c06deaee2, user_id=4fcbac00e1cc4609987d843bd35858b0, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:55.864921 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d815ab6-0b36-4f6f-94c8-ef62836a16f3 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:55.882956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1287/2618] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:55 2022] PUT /identity/v3/projects/17fc95390f2e4d68ac41d16c06deaee2/users/4fcbac00e1cc4609987d843bd35858b0/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:55.886887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f2e35-da86-4510-998c-fc5b3da36bbe admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:55.898469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f2e35-da86-4510-998c-fc5b3da36bbe admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:55.991918 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985b65a-3061-4ccd-bcc9-ba5265484279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:55.992600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985b65a-3061-4ccd-bcc9-ba5265484279 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:55.992880 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985b65a-3061-4ccd-bcc9-ba5265484279 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:55.993151 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985b65a-3061-4ccd-bcc9-ba5265484279 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:55.993912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985b65a-3061-4ccd-bcc9-ba5265484279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:55.996081 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985b65a-3061-4ccd-bcc9-ba5265484279 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:56.018087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1332/2619] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:36:56.022555 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d815ab6-0b36-4f6f-94c8-ef62836a16f3 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:56.041765 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d815ab6-0b36-4f6f-94c8-ef62836a16f3 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:56.047884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b8ae0-362a-4e49-bdf2-4afac039cae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:56.048818 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b8ae0-362a-4e49-bdf2-4afac039cae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:56.049495 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b8ae0-362a-4e49-bdf2-4afac039cae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:56.049788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b8ae0-362a-4e49-bdf2-4afac039cae8 admin admin] PATH_INFO: `/v3/projects/17fc95390f2e4d68ac41d16c06deaee2/users/4fcbac00e1cc4609987d843bd35858b0/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:56.051972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b8ae0-362a-4e49-bdf2-4afac039cae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17fc95390f2e4d68ac41d16c06deaee2, user_id=4fcbac00e1cc4609987d843bd35858b0, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:56.054826 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b8ae0-362a-4e49-bdf2-4afac039cae8 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:56.078863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1288/2620] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:36:56 2022] PUT /identity/v3/projects/17fc95390f2e4d68ac41d16c06deaee2/users/4fcbac00e1cc4609987d843bd35858b0/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:36:56.090360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2bb92-8c93-43cb-b2e3-074165ef67c3 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:56.090939 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2bb92-8c93-43cb-b2e3-074165ef67c3 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:56.091260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2bb92-8c93-43cb-b2e3-074165ef67c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:56.131575 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-3eb2bb92-8c93-43cb-b2e3-074165ef67c3 None None] MFA Rules not processed for user `4fcbac00e1cc4609987d843bd35858b0`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:36:56.132828 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3eb2bb92-8c93-43cb-b2e3-074165ef67c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:56.283865 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1333/2621] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:36:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:36:56.300339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b8ae0-362a-4e49-bdf2-4afac039cae8 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:56.300339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f12b8ae0-362a-4e49-bdf2-4afac039cae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:56.404964 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b8ae0-362a-4e49-bdf2-4afac039cae8 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:56.410447 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a722d-7162-4ead-b95c-d03515b5f77a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:56.411377 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a722d-7162-4ead-b95c-d03515b5f77a service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:56.411865 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a722d-7162-4ead-b95c-d03515b5f77a service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:56.412445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a722d-7162-4ead-b95c-d03515b5f77a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:56.425494 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a722d-7162-4ead-b95c-d03515b5f77a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:56.427534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a722d-7162-4ead-b95c-d03515b5f77a service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:56.447501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1289/2622] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:36:56.517991 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2bb92-8c93-43cb-b2e3-074165ef67c3 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:36:56.521058 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3eb2bb92-8c93-43cb-b2e3-074165ef67c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:36:56.536281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2bb92-8c93-43cb-b2e3-074165ef67c3 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:36:56.541464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9735b3-a25f-4da8-9d92-be5d61dca88f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:36:56.542387 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9735b3-a25f-4da8-9d92-be5d61dca88f service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:36:56.542922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9735b3-a25f-4da8-9d92-be5d61dca88f service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:36:56.543457 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9735b3-a25f-4da8-9d92-be5d61dca88f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:36:56.637531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9735b3-a25f-4da8-9d92-be5d61dca88f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:36:56.639886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9735b3-a25f-4da8-9d92-be5d61dca88f service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:36:56.653986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1334/2623] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:36:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:00.951838 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a722d-7162-4ead-b95c-d03515b5f77a service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:01.101583 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a722d-7162-4ead-b95c-d03515b5f77a service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:01.111386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df869415-e066-451b-b736-be34117446b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:01.112448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df869415-e066-451b-b736-be34117446b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:01.112836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df869415-e066-451b-b736-be34117446b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:01.113784 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df869415-e066-451b-b736-be34117446b5 admin admin] PATH_INFO: `/v3/users/37a796c239c84047a3d6ea77f478ed7b` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:01.116245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df869415-e066-451b-b736-be34117446b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37a796c239c84047a3d6ea77f478ed7b)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:01.120056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df869415-e066-451b-b736-be34117446b5 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:01.227722 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669aec1-3d59-41a4-9072-8adc73be90a3 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:01.228282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669aec1-3d59-41a4-9072-8adc73be90a3 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:01.228438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669aec1-3d59-41a4-9072-8adc73be90a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:01.261488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-0669aec1-3d59-41a4-9072-8adc73be90a3 None None] MFA Rules not processed for user `7cacb076763149cfab7464b1fbbe365d`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:01.262993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-0669aec1-3d59-41a4-9072-8adc73be90a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:01.279641 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-df869415-e066-451b-b736-be34117446b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37a796c239c84047a3d6ea77f478ed7b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:01.295018 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-df869415-e066-451b-b736-be34117446b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37a796c239c84047a3d6ea77f478ed7b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:01.297484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-df869415-e066-451b-b736-be34117446b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37a796c239c84047a3d6ea77f478ed7b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:01.316299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-df869415-e066-451b-b736-be34117446b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37a796c239c84047a3d6ea77f478ed7b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:01.317626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-df869415-e066-451b-b736-be34117446b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37a796c239c84047a3d6ea77f478ed7b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:01.344649 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1290/2624] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:37:00 2022] DELETE /identity/v3/users/37a796c239c84047a3d6ea77f478ed7b => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:01.349357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcee32-6834-4118-9cf0-9964af164198 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:01.349652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcee32-6834-4118-9cf0-9964af164198 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:01.349933 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcee32-6834-4118-9cf0-9964af164198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:01.379093 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1335/2625] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:01 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:01.382787 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-4ffcee32-6834-4118-9cf0-9964af164198 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:01.384043 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-4ffcee32-6834-4118-9cf0-9964af164198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:01.398238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669aec1-3d59-41a4-9072-8adc73be90a3 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:01.399964 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-0669aec1-3d59-41a4-9072-8adc73be90a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:01.477659 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1291/2626] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:37:01.488345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcee32-6834-4118-9cf0-9964af164198 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:01.490852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-4ffcee32-6834-4118-9cf0-9964af164198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:01.555756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669aec1-3d59-41a4-9072-8adc73be90a3 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:01.601583 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcee32-6834-4118-9cf0-9964af164198 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:01.609563 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024559f1-a540-40ac-b289-b62ebf5d8bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:01.610240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024559f1-a540-40ac-b289-b62ebf5d8bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:01.610528 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024559f1-a540-40ac-b289-b62ebf5d8bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:01.610986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024559f1-a540-40ac-b289-b62ebf5d8bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:01.630378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024559f1-a540-40ac-b289-b62ebf5d8bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:01.632431 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024559f1-a540-40ac-b289-b62ebf5d8bf2 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:01.654939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1292/2627] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:37:01.655549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c27296-b051-4965-b4be-e749237f8df0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:01.656220 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c27296-b051-4965-b4be-e749237f8df0 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:01.656621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c27296-b051-4965-b4be-e749237f8df0 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:01.656895 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c27296-b051-4965-b4be-e749237f8df0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:01.659270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-32c27296-b051-4965-b4be-e749237f8df0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:01.774579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c27296-b051-4965-b4be-e749237f8df0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:01.776367 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c27296-b051-4965-b4be-e749237f8df0 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:01.797845 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1336/2628] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:02.246483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024559f1-a540-40ac-b289-b62ebf5d8bf2 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:02.248261 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-024559f1-a540-40ac-b289-b62ebf5d8bf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:02.271264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024559f1-a540-40ac-b289-b62ebf5d8bf2 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:02.276857 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:02.277746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:02.278030 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:02.278309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] PATH_INFO: `/v3/projects/cf022dc7daa24b4b8cbeae7ee9aec6a1` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:02.279713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf022dc7daa24b4b8cbeae7ee9aec6a1)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:02.281655 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:02.365441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf022dc7daa24b4b8cbeae7ee9aec6a1', 'request_id': 'req-293e14c0-5f90-45b1-84f6-c9a103edcce6'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:02.373301 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf022dc7daa24b4b8cbeae7ee9aec6a1', 'request_id': 'req-293e14c0-5f90-45b1-84f6-c9a103edcce6'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:02.387673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf022dc7daa24b4b8cbeae7ee9aec6a1', 'request_id': 'req-293e14c0-5f90-45b1-84f6-c9a103edcce6'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:02.400971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] The token cache is being invalidate because project cf022dc7daa24b4b8cbeae7ee9aec6a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:37:02.401269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:02.402123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:02.407269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1293/2629] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:37:02 2022] DELETE /identity/v3/projects/cf022dc7daa24b4b8cbeae7ee9aec6a1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:02.410048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c27296-b051-4965-b4be-e749237f8df0 service nova] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:02.425635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c27296-b051-4965-b4be-e749237f8df0 service nova] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:02.508219 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2f899-79ea-4b55-a504-0ca7ea8305d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:02.509151 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2f899-79ea-4b55-a504-0ca7ea8305d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:02.509442 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2f899-79ea-4b55-a504-0ca7ea8305d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:02.509833 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2f899-79ea-4b55-a504-0ca7ea8305d3 admin admin] PATH_INFO: `/v3/users/4fcbac00e1cc4609987d843bd35858b0` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:02.513333 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2f899-79ea-4b55-a504-0ca7ea8305d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fcbac00e1cc4609987d843bd35858b0)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:02.515922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2f899-79ea-4b55-a504-0ca7ea8305d3 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:02.585689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-49d2f899-79ea-4b55-a504-0ca7ea8305d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fcbac00e1cc4609987d843bd35858b0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:02.591553 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-49d2f899-79ea-4b55-a504-0ca7ea8305d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fcbac00e1cc4609987d843bd35858b0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:02.605036 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-49d2f899-79ea-4b55-a504-0ca7ea8305d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fcbac00e1cc4609987d843bd35858b0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:02.638728 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-49d2f899-79ea-4b55-a504-0ca7ea8305d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fcbac00e1cc4609987d843bd35858b0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:02.639995 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-49d2f899-79ea-4b55-a504-0ca7ea8305d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fcbac00e1cc4609987d843bd35858b0'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:02.667025 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1337/2630] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:37:02 2022] DELETE /identity/v3/users/4fcbac00e1cc4609987d843bd35858b0 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:03.079618 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:03.081471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:03.098987 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e14c0-5f90-45b1-84f6-c9a103edcce6 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:03.181711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b75b73-9139-4421-8d01-033f45fabd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:03.182882 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b75b73-9139-4421-8d01-033f45fabd58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:03.183294 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b75b73-9139-4421-8d01-033f45fabd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:03.183822 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b75b73-9139-4421-8d01-033f45fabd58 admin admin] PATH_INFO: `/v3/projects/17fc95390f2e4d68ac41d16c06deaee2` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:03.184919 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b75b73-9139-4421-8d01-033f45fabd58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17fc95390f2e4d68ac41d16c06deaee2)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:03.186863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b75b73-9139-4421-8d01-033f45fabd58 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:03.246573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-96b75b73-9139-4421-8d01-033f45fabd58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17fc95390f2e4d68ac41d16c06deaee2', 'request_id': 'req-96b75b73-9139-4421-8d01-033f45fabd58'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:03.252531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-96b75b73-9139-4421-8d01-033f45fabd58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17fc95390f2e4d68ac41d16c06deaee2', 'request_id': 'req-96b75b73-9139-4421-8d01-033f45fabd58'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:03.262358 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-96b75b73-9139-4421-8d01-033f45fabd58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17fc95390f2e4d68ac41d16c06deaee2', 'request_id': 'req-96b75b73-9139-4421-8d01-033f45fabd58'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:03.279034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-96b75b73-9139-4421-8d01-033f45fabd58 admin admin] The token cache is being invalidate because project 17fc95390f2e4d68ac41d16c06deaee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:37:03.279360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-96b75b73-9139-4421-8d01-033f45fabd58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:03.280076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-96b75b73-9139-4421-8d01-033f45fabd58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:03.285313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1294/2631] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:37:03 2022] DELETE /identity/v3/projects/17fc95390f2e4d68ac41d16c06deaee2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:03.287584 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccebe1b-62ba-48b9-9dc0-a9149294f76b None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:03.288216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccebe1b-62ba-48b9-9dc0-a9149294f76b None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:03.288743 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccebe1b-62ba-48b9-9dc0-a9149294f76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:03.315095 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-cccebe1b-62ba-48b9-9dc0-a9149294f76b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:03.316420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-cccebe1b-62ba-48b9-9dc0-a9149294f76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:03.397190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1338/2632] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:37:03.404751 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c571fbe-86d6-47e5-bef2-4f177501db9e None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:03.405039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c571fbe-86d6-47e5-bef2-4f177501db9e None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:03.405315 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c571fbe-86d6-47e5-bef2-4f177501db9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:03.433852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-9c571fbe-86d6-47e5-bef2-4f177501db9e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:03.435113 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9c571fbe-86d6-47e5-bef2-4f177501db9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:03.447087 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1295/2633] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:03.451349 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccebe1b-62ba-48b9-9dc0-a9149294f76b None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:03.462717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccebe1b-62ba-48b9-9dc0-a9149294f76b None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:03.480018 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3926f24-d4da-4180-8968-2e32e10a88df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:03.480744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3926f24-d4da-4180-8968-2e32e10a88df admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:03.481115 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3926f24-d4da-4180-8968-2e32e10a88df admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:03.481418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3926f24-d4da-4180-8968-2e32e10a88df admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:03.482157 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3926f24-d4da-4180-8968-2e32e10a88df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:37:03.482469 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3926f24-d4da-4180-8968-2e32e10a88df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:03.484097 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3926f24-d4da-4180-8968-2e32e10a88df admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:03.504092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1339/2634] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:37:03.509090 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c571fbe-86d6-47e5-bef2-4f177501db9e None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:03.520727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c571fbe-86d6-47e5-bef2-4f177501db9e None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:03.530899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a467c2-d21c-459b-8a95-4958ae87d607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:03.531464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a467c2-d21c-459b-8a95-4958ae87d607 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:03.532439 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a467c2-d21c-459b-8a95-4958ae87d607 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:03.532875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a467c2-d21c-459b-8a95-4958ae87d607 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:03.541162 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a467c2-d21c-459b-8a95-4958ae87d607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:03.543560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a467c2-d21c-459b-8a95-4958ae87d607 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:03.578766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1296/2635] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:37:03 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:03.583326 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3926f24-d4da-4180-8968-2e32e10a88df admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:03.597074 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3926f24-d4da-4180-8968-2e32e10a88df admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:03.682435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc167700-3bb1-4e55-93eb-b28b43672918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:03.683446 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc167700-3bb1-4e55-93eb-b28b43672918 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:03.683804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc167700-3bb1-4e55-93eb-b28b43672918 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:03.684160 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc167700-3bb1-4e55-93eb-b28b43672918 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:03.684922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc167700-3bb1-4e55-93eb-b28b43672918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:03.687000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc167700-3bb1-4e55-93eb-b28b43672918 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:03.719291 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a467c2-d21c-459b-8a95-4958ae87d607 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:03.719617 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1340/2636] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:37:03 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:03.734711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a467c2-d21c-459b-8a95-4958ae87d607 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:03.745461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56744671-3f81-45bc-bad5-a4790d24d153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:03.746437 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56744671-3f81-45bc-bad5-a4790d24d153 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:03.746936 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56744671-3f81-45bc-bad5-a4790d24d153 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:03.747399 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56744671-3f81-45bc-bad5-a4790d24d153 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:03.748296 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56744671-3f81-45bc-bad5-a4790d24d153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:03.750024 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56744671-3f81-45bc-bad5-a4790d24d153 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:03.776247 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1297/2637] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:03.781450 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc167700-3bb1-4e55-93eb-b28b43672918 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:03.794271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc167700-3bb1-4e55-93eb-b28b43672918 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:03.800155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53960ebc-60a9-4f53-b102-e8b0cfecea93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:03.803142 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53960ebc-60a9-4f53-b102-e8b0cfecea93 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:03.803454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53960ebc-60a9-4f53-b102-e8b0cfecea93 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:03.804783 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53960ebc-60a9-4f53-b102-e8b0cfecea93 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:03.805587 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53960ebc-60a9-4f53-b102-e8b0cfecea93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:03.808060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53960ebc-60a9-4f53-b102-e8b0cfecea93 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:03.824281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1341/2638] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:03.828806 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56744671-3f81-45bc-bad5-a4790d24d153 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:03.839966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56744671-3f81-45bc-bad5-a4790d24d153 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:03.845737 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3dfb7-c1e0-4466-a5d6-7d58afbc946b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:03.846901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3dfb7-c1e0-4466-a5d6-7d58afbc946b admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:03.847802 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3dfb7-c1e0-4466-a5d6-7d58afbc946b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:03.848314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3dfb7-c1e0-4466-a5d6-7d58afbc946b admin admin] PATH_INFO: `/v3/projects/0ebc9ee7a9b048dba7e41c9569ec27d7/users/1384b7262acc4cabba80be590c38f569/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:03.889417 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3dfb7-c1e0-4466-a5d6-7d58afbc946b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ebc9ee7a9b048dba7e41c9569ec27d7, user_id=1384b7262acc4cabba80be590c38f569, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:03.897851 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3dfb7-c1e0-4466-a5d6-7d58afbc946b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:03.921675 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1298/2639] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:37:03 2022] PUT /identity/v3/projects/0ebc9ee7a9b048dba7e41c9569ec27d7/users/1384b7262acc4cabba80be590c38f569/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:03.932927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040f535-67aa-4692-9fa4-d2b318d57b21 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:03.933541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040f535-67aa-4692-9fa4-d2b318d57b21 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:03.933916 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040f535-67aa-4692-9fa4-d2b318d57b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:03.972798 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-6040f535-67aa-4692-9fa4-d2b318d57b21 None None] MFA Rules not processed for user `1384b7262acc4cabba80be590c38f569`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:03.976942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-6040f535-67aa-4692-9fa4-d2b318d57b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:04.050975 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1342/2640] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:37:04.055254 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3dfb7-c1e0-4466-a5d6-7d58afbc946b admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:04.152695 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3dfb7-c1e0-4466-a5d6-7d58afbc946b admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:04.158207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb25fa1-e132-45e7-a9ca-4a8f5b66c507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:04.160072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb25fa1-e132-45e7-a9ca-4a8f5b66c507 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:04.160072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb25fa1-e132-45e7-a9ca-4a8f5b66c507 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:04.160072 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb25fa1-e132-45e7-a9ca-4a8f5b66c507 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:04.160366 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb25fa1-e132-45e7-a9ca-4a8f5b66c507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:04.162287 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb25fa1-e132-45e7-a9ca-4a8f5b66c507 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:04.196491 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1299/2641] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:37:04 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:04.199400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040f535-67aa-4692-9fa4-d2b318d57b21 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:04.212375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040f535-67aa-4692-9fa4-d2b318d57b21 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:04.280441 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5a0a9-5fe2-4bdf-b203-cb136cb97aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:04.280985 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5a0a9-5fe2-4bdf-b203-cb136cb97aba admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:04.281270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5a0a9-5fe2-4bdf-b203-cb136cb97aba admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:04.281545 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5a0a9-5fe2-4bdf-b203-cb136cb97aba admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:04.282224 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5a0a9-5fe2-4bdf-b203-cb136cb97aba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:04.284483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5a0a9-5fe2-4bdf-b203-cb136cb97aba admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:04.305694 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1343/2642] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:37:04 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:04.309350 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb25fa1-e132-45e7-a9ca-4a8f5b66c507 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:04.320188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb25fa1-e132-45e7-a9ca-4a8f5b66c507 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:04.325032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c5195-157a-438d-a7e6-0508d57cbf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:04.325707 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c5195-157a-438d-a7e6-0508d57cbf3d admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:04.325992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c5195-157a-438d-a7e6-0508d57cbf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:04.326333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c5195-157a-438d-a7e6-0508d57cbf3d admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:04.327114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c5195-157a-438d-a7e6-0508d57cbf3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:04.332681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c5195-157a-438d-a7e6-0508d57cbf3d admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:04.344598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1300/2643] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:37:04.349424 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5a0a9-5fe2-4bdf-b203-cb136cb97aba admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:04.363377 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5a0a9-5fe2-4bdf-b203-cb136cb97aba admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:04.368566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443ba3a-cc84-400d-acd7-38bcc84f710a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:04.369560 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443ba3a-cc84-400d-acd7-38bcc84f710a admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:04.369876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443ba3a-cc84-400d-acd7-38bcc84f710a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:04.370408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443ba3a-cc84-400d-acd7-38bcc84f710a admin admin] PATH_INFO: `/v3/projects/63c7c828b0de4b63bfdfa1f0f3695d82/users/d976c79aa2d84f98911a8683b1a82c06/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:04.389108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443ba3a-cc84-400d-acd7-38bcc84f710a admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c7c828b0de4b63bfdfa1f0f3695d82, user_id=d976c79aa2d84f98911a8683b1a82c06, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:04.391823 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443ba3a-cc84-400d-acd7-38bcc84f710a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:04.407015 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1344/2644] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:37:04 2022] PUT /identity/v3/projects/63c7c828b0de4b63bfdfa1f0f3695d82/users/d976c79aa2d84f98911a8683b1a82c06/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:37:04.410968 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c5195-157a-438d-a7e6-0508d57cbf3d admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:04.420995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c5195-157a-438d-a7e6-0508d57cbf3d admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:04.483750 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0fb17-da22-4c39-9535-2391c650d5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:04.484420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0fb17-da22-4c39-9535-2391c650d5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:04.484698 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0fb17-da22-4c39-9535-2391c650d5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:04.484986 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0fb17-da22-4c39-9535-2391c650d5a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:04.485720 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0fb17-da22-4c39-9535-2391c650d5a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:04.487319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0fb17-da22-4c39-9535-2391c650d5a9 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:04.501040 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1301/2645] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:04.506977 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443ba3a-cc84-400d-acd7-38bcc84f710a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:04.522365 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443ba3a-cc84-400d-acd7-38bcc84f710a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:04.527070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ed9d9-44b6-4daf-a743-2690d2494199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:04.528498 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ed9d9-44b6-4daf-a743-2690d2494199 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:04.529000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ed9d9-44b6-4daf-a743-2690d2494199 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:04.529500 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ed9d9-44b6-4daf-a743-2690d2494199 admin admin] PATH_INFO: `/v3/projects/63c7c828b0de4b63bfdfa1f0f3695d82/users/d976c79aa2d84f98911a8683b1a82c06/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:04.532466 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ed9d9-44b6-4daf-a743-2690d2494199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c7c828b0de4b63bfdfa1f0f3695d82, user_id=d976c79aa2d84f98911a8683b1a82c06, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:04.535882 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ed9d9-44b6-4daf-a743-2690d2494199 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:04.550291 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1345/2646] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:37:04 2022] PUT /identity/v3/projects/63c7c828b0de4b63bfdfa1f0f3695d82/users/d976c79aa2d84f98911a8683b1a82c06/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:04.560424 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbbcfc-f8a8-46e4-9c50-c737220f3561 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:04.560710 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbbcfc-f8a8-46e4-9c50-c737220f3561 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:04.561236 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbbcfc-f8a8-46e4-9c50-c737220f3561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:04.593098 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-f9fbbcfc-f8a8-46e4-9c50-c737220f3561 None None] MFA Rules not processed for user `d976c79aa2d84f98911a8683b1a82c06`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:04.594292 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f9fbbcfc-f8a8-46e4-9c50-c737220f3561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:04.665331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1302/2647] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:04.677090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ed9d9-44b6-4daf-a743-2690d2494199 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:04.678920 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-df4ed9d9-44b6-4daf-a743-2690d2494199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:04.735526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ed9d9-44b6-4daf-a743-2690d2494199 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:04.739276 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87deaefd-d08b-46c3-9fb1-aa833faec405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:04.739948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87deaefd-d08b-46c3-9fb1-aa833faec405 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:04.740228 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87deaefd-d08b-46c3-9fb1-aa833faec405 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:04.740523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87deaefd-d08b-46c3-9fb1-aa833faec405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:04.750884 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87deaefd-d08b-46c3-9fb1-aa833faec405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:04.752339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87deaefd-d08b-46c3-9fb1-aa833faec405 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:04.762318 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1346/2648] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:04.857597 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbbcfc-f8a8-46e4-9c50-c737220f3561 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:04.868022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbbcfc-f8a8-46e4-9c50-c737220f3561 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:04.871939 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92638894-84d9-4759-98e4-f42e0e00e51f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:04.872587 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92638894-84d9-4759-98e4-f42e0e00e51f service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:04.872870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92638894-84d9-4759-98e4-f42e0e00e51f service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:04.873143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92638894-84d9-4759-98e4-f42e0e00e51f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:04.946217 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92638894-84d9-4759-98e4-f42e0e00e51f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:04.946217 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92638894-84d9-4759-98e4-f42e0e00e51f service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:04.954792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1303/2649] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:05.292793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87deaefd-d08b-46c3-9fb1-aa833faec405 service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:05.420544 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87deaefd-d08b-46c3-9fb1-aa833faec405 service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:05.426147 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b86a0-aaf0-4f25-829b-f23a7ef3c007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:05.427389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b86a0-aaf0-4f25-829b-f23a7ef3c007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:05.428035 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b86a0-aaf0-4f25-829b-f23a7ef3c007 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:05.428393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b86a0-aaf0-4f25-829b-f23a7ef3c007 admin admin] PATH_INFO: `/v3/users/1384b7262acc4cabba80be590c38f569` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:05.430041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b86a0-aaf0-4f25-829b-f23a7ef3c007 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1384b7262acc4cabba80be590c38f569)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:05.432868 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b86a0-aaf0-4f25-829b-f23a7ef3c007 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:05.507413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-632b86a0-aaf0-4f25-829b-f23a7ef3c007 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1384b7262acc4cabba80be590c38f569'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:05.508791 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-632b86a0-aaf0-4f25-829b-f23a7ef3c007 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1384b7262acc4cabba80be590c38f569'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:05.518479 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-632b86a0-aaf0-4f25-829b-f23a7ef3c007 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1384b7262acc4cabba80be590c38f569'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:05.546078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-632b86a0-aaf0-4f25-829b-f23a7ef3c007 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1384b7262acc4cabba80be590c38f569'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:05.547301 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-632b86a0-aaf0-4f25-829b-f23a7ef3c007 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1384b7262acc4cabba80be590c38f569'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:05.572357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1347/2650] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:37:05 2022] DELETE /identity/v3/users/1384b7262acc4cabba80be590c38f569 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:05.576651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862bde-6bec-4af7-927f-2b386a6d2a72 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:05.578289 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862bde-6bec-4af7-927f-2b386a6d2a72 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:05.578811 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862bde-6bec-4af7-927f-2b386a6d2a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:05.599379 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-0d862bde-6bec-4af7-927f-2b386a6d2a72 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:05.600959 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-0d862bde-6bec-4af7-927f-2b386a6d2a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:05.664963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1304/2651] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:37:05.679051 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b86a0-aaf0-4f25-829b-f23a7ef3c007 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:05.679051 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-632b86a0-aaf0-4f25-829b-f23a7ef3c007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:05.743490 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b86a0-aaf0-4f25-829b-f23a7ef3c007 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:05.748618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b48a8-2fc1-4350-bcda-181e531d5393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:05.749909 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b48a8-2fc1-4350-bcda-181e531d5393 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:05.750204 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b48a8-2fc1-4350-bcda-181e531d5393 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:05.750480 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b48a8-2fc1-4350-bcda-181e531d5393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:05.764339 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b48a8-2fc1-4350-bcda-181e531d5393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:05.765884 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b48a8-2fc1-4350-bcda-181e531d5393 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:05.778316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1348/2652] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:06.099965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d862bde-6bec-4af7-927f-2b386a6d2a72 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:06.101941 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-0d862bde-6bec-4af7-927f-2b386a6d2a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:06.117180 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d862bde-6bec-4af7-927f-2b386a6d2a72 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:06.123614 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:06.125135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:06.125487 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:06.125846 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] PATH_INFO: `/v3/projects/0ebc9ee7a9b048dba7e41c9569ec27d7` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:06.127152 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ebc9ee7a9b048dba7e41c9569ec27d7)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:06.129143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:06.199211 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ebc9ee7a9b048dba7e41c9569ec27d7', 'request_id': 'req-4041ce18-38a4-4af6-96eb-0079e32b75fc'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.208935 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ebc9ee7a9b048dba7e41c9569ec27d7', 'request_id': 'req-4041ce18-38a4-4af6-96eb-0079e32b75fc'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.217955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ebc9ee7a9b048dba7e41c9569ec27d7', 'request_id': 'req-4041ce18-38a4-4af6-96eb-0079e32b75fc'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.230559 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] The token cache is being invalidate because project 0ebc9ee7a9b048dba7e41c9569ec27d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:37:06.231220 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.232111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.233678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1305/2653] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:37:06 2022] DELETE /identity/v3/projects/0ebc9ee7a9b048dba7e41c9569ec27d7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:06.237636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b48a8-2fc1-4350-bcda-181e531d5393 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:06.250530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b48a8-2fc1-4350-bcda-181e531d5393 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:06.316044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac9f93-02b8-4f10-9536-6f95484a2c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:06.317227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac9f93-02b8-4f10-9536-6f95484a2c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:06.317783 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac9f93-02b8-4f10-9536-6f95484a2c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:06.318158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac9f93-02b8-4f10-9536-6f95484a2c6e admin admin] PATH_INFO: `/v3/users/d976c79aa2d84f98911a8683b1a82c06` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:06.319634 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac9f93-02b8-4f10-9536-6f95484a2c6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d976c79aa2d84f98911a8683b1a82c06)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:06.321709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac9f93-02b8-4f10-9536-6f95484a2c6e admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:06.386925 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8dac9f93-02b8-4f10-9536-6f95484a2c6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd976c79aa2d84f98911a8683b1a82c06'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.387640 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8dac9f93-02b8-4f10-9536-6f95484a2c6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd976c79aa2d84f98911a8683b1a82c06'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.401832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8dac9f93-02b8-4f10-9536-6f95484a2c6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd976c79aa2d84f98911a8683b1a82c06'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.409747 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8dac9f93-02b8-4f10-9536-6f95484a2c6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd976c79aa2d84f98911a8683b1a82c06'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.410603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8dac9f93-02b8-4f10-9536-6f95484a2c6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd976c79aa2d84f98911a8683b1a82c06'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.431787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1349/2654] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:37:06 2022] DELETE /identity/v3/users/d976c79aa2d84f98911a8683b1a82c06 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:06.760517 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:06.762255 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:06.799306 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041ce18-38a4-4af6-96eb-0079e32b75fc admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:06.878781 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f24e53-2437-4baa-a248-1f55e621934d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:06.879693 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f24e53-2437-4baa-a248-1f55e621934d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:06.880001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f24e53-2437-4baa-a248-1f55e621934d admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:06.880275 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f24e53-2437-4baa-a248-1f55e621934d admin admin] PATH_INFO: `/v3/projects/63c7c828b0de4b63bfdfa1f0f3695d82` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:06.881482 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f24e53-2437-4baa-a248-1f55e621934d admin admin] RBAC: Authorizing `identity:delete_project(project_id=63c7c828b0de4b63bfdfa1f0f3695d82)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:06.883504 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f24e53-2437-4baa-a248-1f55e621934d admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:06.928545 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d0f24e53-2437-4baa-a248-1f55e621934d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63c7c828b0de4b63bfdfa1f0f3695d82', 'request_id': 'req-d0f24e53-2437-4baa-a248-1f55e621934d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.933836 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d0f24e53-2437-4baa-a248-1f55e621934d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63c7c828b0de4b63bfdfa1f0f3695d82', 'request_id': 'req-d0f24e53-2437-4baa-a248-1f55e621934d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.941159 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d0f24e53-2437-4baa-a248-1f55e621934d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63c7c828b0de4b63bfdfa1f0f3695d82', 'request_id': 'req-d0f24e53-2437-4baa-a248-1f55e621934d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.953682 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d0f24e53-2437-4baa-a248-1f55e621934d admin admin] The token cache is being invalidate because project 63c7c828b0de4b63bfdfa1f0f3695d82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:37:06.953975 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d0f24e53-2437-4baa-a248-1f55e621934d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.954628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-d0f24e53-2437-4baa-a248-1f55e621934d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:06.956245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1306/2655] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:37:06 2022] DELETE /identity/v3/projects/63c7c828b0de4b63bfdfa1f0f3695d82 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:06.961191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45c51f-a4f4-4f82-bd3f-84f81badd8c7 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:06.961732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45c51f-a4f4-4f82-bd3f-84f81badd8c7 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:06.961845 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45c51f-a4f4-4f82-bd3f-84f81badd8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:06.981475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4a45c51f-a4f4-4f82-bd3f-84f81badd8c7 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:06.982706 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4a45c51f-a4f4-4f82-bd3f-84f81badd8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:07.043689 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1350/2656] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:07.053228 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44aadaf-8163-43ee-befb-e199cc28653f None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:07.053549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44aadaf-8163-43ee-befb-e199cc28653f None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:07.053831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44aadaf-8163-43ee-befb-e199cc28653f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:07.072875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-e44aadaf-8163-43ee-befb-e199cc28653f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:07.074322 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e44aadaf-8163-43ee-befb-e199cc28653f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:07.084471 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1307/2657] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:07.088827 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45c51f-a4f4-4f82-bd3f-84f81badd8c7 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:07.098216 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45c51f-a4f4-4f82-bd3f-84f81badd8c7 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:07.103527 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37760a-9d9b-49fd-b782-2af23d2309b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:07.104197 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37760a-9d9b-49fd-b782-2af23d2309b5 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:07.104476 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37760a-9d9b-49fd-b782-2af23d2309b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:07.104749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37760a-9d9b-49fd-b782-2af23d2309b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:07.105483 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37760a-9d9b-49fd-b782-2af23d2309b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:37:07.105788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37760a-9d9b-49fd-b782-2af23d2309b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:07.107432 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37760a-9d9b-49fd-b782-2af23d2309b5 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:07.119702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1351/2658] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:37:07.124267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44aadaf-8163-43ee-befb-e199cc28653f None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:07.133362 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44aadaf-8163-43ee-befb-e199cc28653f None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:07.137965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0a0d2-1b98-4e87-9cd7-6e858d30d7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:07.138606 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0a0d2-1b98-4e87-9cd7-6e858d30d7e9 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:07.138884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0a0d2-1b98-4e87-9cd7-6e858d30d7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:07.139156 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0a0d2-1b98-4e87-9cd7-6e858d30d7e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:07.139901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0a0d2-1b98-4e87-9cd7-6e858d30d7e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:07.142194 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0a0d2-1b98-4e87-9cd7-6e858d30d7e9 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:07.169050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1308/2659] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:37:07 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:37:07.173117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37760a-9d9b-49fd-b782-2af23d2309b5 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:07.183158 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37760a-9d9b-49fd-b782-2af23d2309b5 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:07.239133 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a468e9c-1f4b-4602-a66c-db876083f32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:07.240248 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a468e9c-1f4b-4602-a66c-db876083f32d admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:07.240742 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a468e9c-1f4b-4602-a66c-db876083f32d admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:07.241024 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a468e9c-1f4b-4602-a66c-db876083f32d admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:07.241703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a468e9c-1f4b-4602-a66c-db876083f32d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:07.243637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a468e9c-1f4b-4602-a66c-db876083f32d admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:07.263297 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1352/2660] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:37:07 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:07.267328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0a0d2-1b98-4e87-9cd7-6e858d30d7e9 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:07.278050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0a0d2-1b98-4e87-9cd7-6e858d30d7e9 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:07.282626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba11df-14f4-4ad5-b2bf-fff649846468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:07.283398 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba11df-14f4-4ad5-b2bf-fff649846468 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:07.283681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba11df-14f4-4ad5-b2bf-fff649846468 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:07.283953 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba11df-14f4-4ad5-b2bf-fff649846468 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:07.284672 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba11df-14f4-4ad5-b2bf-fff649846468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:07.286991 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba11df-14f4-4ad5-b2bf-fff649846468 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:07.297116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1309/2661] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:07.300841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a468e9c-1f4b-4602-a66c-db876083f32d admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:07.310594 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a468e9c-1f4b-4602-a66c-db876083f32d admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:07.315347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468cca9-e1e2-44be-b6ec-99e5edc8751f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:07.316014 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468cca9-e1e2-44be-b6ec-99e5edc8751f admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:07.316352 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468cca9-e1e2-44be-b6ec-99e5edc8751f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:07.316639 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468cca9-e1e2-44be-b6ec-99e5edc8751f admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:07.317366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468cca9-e1e2-44be-b6ec-99e5edc8751f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:07.318958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468cca9-e1e2-44be-b6ec-99e5edc8751f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:07.329062 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1353/2662] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:37:07.332924 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba11df-14f4-4ad5-b2bf-fff649846468 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:07.343369 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba11df-14f4-4ad5-b2bf-fff649846468 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:07.348097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cff6c3-f203-4294-b968-e0adec38029f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:07.349600 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cff6c3-f203-4294-b968-e0adec38029f admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:07.349883 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cff6c3-f203-4294-b968-e0adec38029f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:07.350159 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cff6c3-f203-4294-b968-e0adec38029f admin admin] PATH_INFO: `/v3/projects/9dfe380cce1e4dc8b016422e4f40f547/users/4b27e8cf716c4c41a994be5addc9b722/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:07.367504 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cff6c3-f203-4294-b968-e0adec38029f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dfe380cce1e4dc8b016422e4f40f547, user_id=4b27e8cf716c4c41a994be5addc9b722, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:07.371055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cff6c3-f203-4294-b968-e0adec38029f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:07.384585 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1310/2663] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:37:07 2022] PUT /identity/v3/projects/9dfe380cce1e4dc8b016422e4f40f547/users/4b27e8cf716c4c41a994be5addc9b722/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:07.396883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a5a03-f6ff-4d74-adcd-7749cc6e4867 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:07.397173 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a5a03-f6ff-4d74-adcd-7749cc6e4867 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:07.397453 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a5a03-f6ff-4d74-adcd-7749cc6e4867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:07.424361 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-947a5a03-f6ff-4d74-adcd-7749cc6e4867 None None] MFA Rules not processed for user `4b27e8cf716c4c41a994be5addc9b722`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:07.425555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-947a5a03-f6ff-4d74-adcd-7749cc6e4867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:07.487420 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1354/2664] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:07.498374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cff6c3-f203-4294-b968-e0adec38029f admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:07.500057 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c9cff6c3-f203-4294-b968-e0adec38029f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:07.555769 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cff6c3-f203-4294-b968-e0adec38029f admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:07.559386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2636a06-cd77-47ca-b7ba-15e05e7a97a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:07.560320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2636a06-cd77-47ca-b7ba-15e05e7a97a6 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:07.560599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2636a06-cd77-47ca-b7ba-15e05e7a97a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:07.560872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2636a06-cd77-47ca-b7ba-15e05e7a97a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:07.569690 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2636a06-cd77-47ca-b7ba-15e05e7a97a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:07.571264 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2636a06-cd77-47ca-b7ba-15e05e7a97a6 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:07.580957 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1311/2665] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:07.660006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a5a03-f6ff-4d74-adcd-7749cc6e4867 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:07.661731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-947a5a03-f6ff-4d74-adcd-7749cc6e4867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:07.718675 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a5a03-f6ff-4d74-adcd-7749cc6e4867 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:07.726670 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb4067-ad78-41e7-b5e5-09801c7c7b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:07.727367 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb4067-ad78-41e7-b5e5-09801c7c7b36 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:07.727654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb4067-ad78-41e7-b5e5-09801c7c7b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:07.727935 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb4067-ad78-41e7-b5e5-09801c7c7b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:07.729906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-8ddb4067-ad78-41e7-b5e5-09801c7c7b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:07.804540 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb4067-ad78-41e7-b5e5-09801c7c7b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:07.806560 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb4067-ad78-41e7-b5e5-09801c7c7b36 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:07.817083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1355/2666] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:09.940122 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f5a4c-093c-407d-ae64-04652a88e45a None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:09.940712 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f5a4c-093c-407d-ae64-04652a88e45a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:09.941790 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f5a4c-093c-407d-ae64-04652a88e45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:09.981560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-232f5a4c-093c-407d-ae64-04652a88e45a None None] MFA Rules not processed for user `b2e8f8cb607649c0836a9c709791fc0e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:09.983216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-232f5a4c-093c-407d-ae64-04652a88e45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:10.080713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1312/2667] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:11.687551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb4067-ad78-41e7-b5e5-09801c7c7b36 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:11.689024 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-8ddb4067-ad78-41e7-b5e5-09801c7c7b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:11.706480 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb4067-ad78-41e7-b5e5-09801c7c7b36 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:11.711896 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd93b8-e7f0-4a70-89e0-f490db3d3229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:11.712770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd93b8-e7f0-4a70-89e0-f490db3d3229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:11.713044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd93b8-e7f0-4a70-89e0-f490db3d3229 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:11.713309 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd93b8-e7f0-4a70-89e0-f490db3d3229 admin admin] PATH_INFO: `/v3/users/cac32da403af402c8236f46168e81038` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:11.714470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd93b8-e7f0-4a70-89e0-f490db3d3229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cac32da403af402c8236f46168e81038)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:11.716407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd93b8-e7f0-4a70-89e0-f490db3d3229 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:11.764075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-dffd93b8-e7f0-4a70-89e0-f490db3d3229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cac32da403af402c8236f46168e81038'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:11.764723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-dffd93b8-e7f0-4a70-89e0-f490db3d3229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cac32da403af402c8236f46168e81038'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:11.772401 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-dffd93b8-e7f0-4a70-89e0-f490db3d3229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cac32da403af402c8236f46168e81038'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:11.779593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-dffd93b8-e7f0-4a70-89e0-f490db3d3229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cac32da403af402c8236f46168e81038'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:11.780337 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-dffd93b8-e7f0-4a70-89e0-f490db3d3229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cac32da403af402c8236f46168e81038'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:11.797206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1356/2668] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:37:11 2022] DELETE /identity/v3/users/cac32da403af402c8236f46168e81038 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:11.933983 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f5a4c-093c-407d-ae64-04652a88e45a None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:11.950347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f5a4c-093c-407d-ae64-04652a88e45a None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:12.052109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58023b64-9e38-4df0-90b7-49dc938d3443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:12.053602 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58023b64-9e38-4df0-90b7-49dc938d3443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.054136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58023b64-9e38-4df0-90b7-49dc938d3443 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.054581 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58023b64-9e38-4df0-90b7-49dc938d3443 admin admin] PATH_INFO: `/v3/users/4b27e8cf716c4c41a994be5addc9b722` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.058303 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58023b64-9e38-4df0-90b7-49dc938d3443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b27e8cf716c4c41a994be5addc9b722)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:12.060930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58023b64-9e38-4df0-90b7-49dc938d3443 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:12.117934 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd93b8-e7f0-4a70-89e0-f490db3d3229 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:12.120853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-dffd93b8-e7f0-4a70-89e0-f490db3d3229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:12.137780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd93b8-e7f0-4a70-89e0-f490db3d3229 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:12.159868 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:12.161335 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.161618 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.161943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae admin admin] PATH_INFO: `/v3/projects/127c608e5ebd478282b045739182c41e` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.163601 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=127c608e5ebd478282b045739182c41e)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:12.166262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:12.169311 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-58023b64-9e38-4df0-90b7-49dc938d3443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b27e8cf716c4c41a994be5addc9b722'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:12.179231 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-58023b64-9e38-4df0-90b7-49dc938d3443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b27e8cf716c4c41a994be5addc9b722'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:12.180390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-58023b64-9e38-4df0-90b7-49dc938d3443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b27e8cf716c4c41a994be5addc9b722'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:12.190483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-58023b64-9e38-4df0-90b7-49dc938d3443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b27e8cf716c4c41a994be5addc9b722'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:12.191135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-58023b64-9e38-4df0-90b7-49dc938d3443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b27e8cf716c4c41a994be5addc9b722'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:12.213443 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1313/2669] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:37:11 2022] DELETE /identity/v3/users/4b27e8cf716c4c41a994be5addc9b722 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:12.218318 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ec90a-23c0-49de-97b6-290977bcf19c None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.218666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ec90a-23c0-49de-97b6-290977bcf19c None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.218925 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ec90a-23c0-49de-97b6-290977bcf19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.236937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-877ec90a-23c0-49de-97b6-290977bcf19c None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:12.238605 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-877ec90a-23c0-49de-97b6-290977bcf19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:12.242045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '127c608e5ebd478282b045739182c41e', 'request_id': 'req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:12.250935 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '127c608e5ebd478282b045739182c41e', 'request_id': 'req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:12.260497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '127c608e5ebd478282b045739182c41e', 'request_id': 'req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:12.275003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae admin admin] The token cache is being invalidate because project 127c608e5ebd478282b045739182c41e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:37:12.275003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:12.276968 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-eb2c5e8a-4ecb-4267-ae90-96ce92dc05ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:12.278799 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1357/2670] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:37:12 2022] DELETE /identity/v3/projects/127c608e5ebd478282b045739182c41e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:12.283562 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17b882-7ee3-4789-b54c-2497602f9a27 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.283851 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17b882-7ee3-4789-b54c-2497602f9a27 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.284186 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17b882-7ee3-4789-b54c-2497602f9a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.302523 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-eb17b882-7ee3-4789-b54c-2497602f9a27 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:12.303809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-eb17b882-7ee3-4789-b54c-2497602f9a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:12.318930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1314/2671] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:12.330445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ec90a-23c0-49de-97b6-290977bcf19c None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:12.332808 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-877ec90a-23c0-49de-97b6-290977bcf19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:12.389913 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1358/2672] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:37:12.396966 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ec90a-23c0-49de-97b6-290977bcf19c None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:12.398165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035e729-e882-4772-9841-d952d9420311 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.398459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035e729-e882-4772-9841-d952d9420311 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.398745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035e729-e882-4772-9841-d952d9420311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.401811 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a113dc-bfe3-4cc9-8b40-0b3681c849ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:12.402798 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a113dc-bfe3-4cc9-8b40-0b3681c849ef service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.403110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a113dc-bfe3-4cc9-8b40-0b3681c849ef service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.403644 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a113dc-bfe3-4cc9-8b40-0b3681c849ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.414920 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a113dc-bfe3-4cc9-8b40-0b3681c849ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:12.416435 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a113dc-bfe3-4cc9-8b40-0b3681c849ef service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:12.426441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1315/2673] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:37:12.432701 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-7035e729-e882-4772-9841-d952d9420311 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:12.433859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7035e729-e882-4772-9841-d952d9420311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:12.443007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1359/2674] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:37:12.447435 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a113dc-bfe3-4cc9-8b40-0b3681c849ef service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:12.458940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a113dc-bfe3-4cc9-8b40-0b3681c849ef service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:12.471326 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f850b-1d35-42ca-a9e2-3e529e90ccae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:12.472770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f850b-1d35-42ca-a9e2-3e529e90ccae admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.473076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f850b-1d35-42ca-a9e2-3e529e90ccae admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.473374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f850b-1d35-42ca-a9e2-3e529e90ccae admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.474184 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f850b-1d35-42ca-a9e2-3e529e90ccae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:37:12.474496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f850b-1d35-42ca-a9e2-3e529e90ccae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:12.476362 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f850b-1d35-42ca-a9e2-3e529e90ccae admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:12.488897 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1316/2675] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:37:12.493860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035e729-e882-4772-9841-d952d9420311 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:12.504171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035e729-e882-4772-9841-d952d9420311 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:12.510260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32663756-5db8-40a2-afab-0f8f1c695f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:12.511280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663756-5db8-40a2-afab-0f8f1c695f98 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.511777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663756-5db8-40a2-afab-0f8f1c695f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.512060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663756-5db8-40a2-afab-0f8f1c695f98 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.512751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32663756-5db8-40a2-afab-0f8f1c695f98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:12.514831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32663756-5db8-40a2-afab-0f8f1c695f98 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:12.559470 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1360/2676] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:37:12 2022] POST /identity/v3/projects => generated 355 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:12.562680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f850b-1d35-42ca-a9e2-3e529e90ccae admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:12.576926 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f850b-1d35-42ca-a9e2-3e529e90ccae admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:12.646109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f66b9c-5af5-498e-92cd-fb20b27c1ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:12.646892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f66b9c-5af5-498e-92cd-fb20b27c1ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.647551 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f66b9c-5af5-498e-92cd-fb20b27c1ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.647551 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f66b9c-5af5-498e-92cd-fb20b27c1ae3 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.648339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f66b9c-5af5-498e-92cd-fb20b27c1ae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:12.650364 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f66b9c-5af5-498e-92cd-fb20b27c1ae3 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:12.677154 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1317/2677] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:37:12 2022] POST /identity/v3/users => generated 278 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:12.678718 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32663756-5db8-40a2-afab-0f8f1c695f98 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:12.689563 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32663756-5db8-40a2-afab-0f8f1c695f98 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:12.695888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c92152-33a2-4727-812f-94c301ca83c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:12.696564 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c92152-33a2-4727-812f-94c301ca83c6 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.696853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c92152-33a2-4727-812f-94c301ca83c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.697130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c92152-33a2-4727-812f-94c301ca83c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.697861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c92152-33a2-4727-812f-94c301ca83c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:12.699494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c92152-33a2-4727-812f-94c301ca83c6 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:12.711328 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1361/2678] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:12.715340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f66b9c-5af5-498e-92cd-fb20b27c1ae3 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:12.727729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f66b9c-5af5-498e-92cd-fb20b27c1ae3 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:12.732971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7f0f7-1fca-42f6-bab7-0b8cad80bdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:12.733681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7f0f7-1fca-42f6-bab7-0b8cad80bdf5 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.733973 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7f0f7-1fca-42f6-bab7-0b8cad80bdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.734249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7f0f7-1fca-42f6-bab7-0b8cad80bdf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.734985 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7f0f7-1fca-42f6-bab7-0b8cad80bdf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:12.736637 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7f0f7-1fca-42f6-bab7-0b8cad80bdf5 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:12.751848 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1318/2679] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:12.753279 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c92152-33a2-4727-812f-94c301ca83c6 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:12.763666 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c92152-33a2-4727-812f-94c301ca83c6 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:12.768488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2faec70-f70d-4041-a5db-739fb0a06569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:12.769380 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faec70-f70d-4041-a5db-739fb0a06569 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.769742 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faec70-f70d-4041-a5db-739fb0a06569 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.770065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faec70-f70d-4041-a5db-739fb0a06569 admin admin] PATH_INFO: `/v3/projects/2f67c315056b44ad98b6143428d77ebd/users/6cdb190d146a48f2891b2a7967ab59c1/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.785494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2faec70-f70d-4041-a5db-739fb0a06569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f67c315056b44ad98b6143428d77ebd, user_id=6cdb190d146a48f2891b2a7967ab59c1, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:12.788771 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2faec70-f70d-4041-a5db-739fb0a06569 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:12.807023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1362/2680] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:37:12 2022] PUT /identity/v3/projects/2f67c315056b44ad98b6143428d77ebd/users/6cdb190d146a48f2891b2a7967ab59c1/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:12.817849 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a5f73-9d98-4084-b93f-e249f54a11c9 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.818237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a5f73-9d98-4084-b93f-e249f54a11c9 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.818563 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a5f73-9d98-4084-b93f-e249f54a11c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.847412 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-7a8a5f73-9d98-4084-b93f-e249f54a11c9 None None] MFA Rules not processed for user `6cdb190d146a48f2891b2a7967ab59c1`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:12.851284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7a8a5f73-9d98-4084-b93f-e249f54a11c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:12.851488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2faec70-f70d-4041-a5db-739fb0a06569 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:12.854310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a2faec70-f70d-4041-a5db-739fb0a06569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:12.933463 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2faec70-f70d-4041-a5db-739fb0a06569 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:12.938536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5303b9-544d-4f72-83c9-d6a6c0e96757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:12.939214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5303b9-544d-4f72-83c9-d6a6c0e96757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.939508 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5303b9-544d-4f72-83c9-d6a6c0e96757 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.939808 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5303b9-544d-4f72-83c9-d6a6c0e96757 admin admin] PATH_INFO: `/v3/projects/9dfe380cce1e4dc8b016422e4f40f547` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.940942 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5303b9-544d-4f72-83c9-d6a6c0e96757 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dfe380cce1e4dc8b016422e4f40f547)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:12.942358 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1319/2681] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:12 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:12.942930 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5303b9-544d-4f72-83c9-d6a6c0e96757 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:12.946545 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a5f73-9d98-4084-b93f-e249f54a11c9 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:12.961666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a5f73-9d98-4084-b93f-e249f54a11c9 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:12.973278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9a542-659e-4e5a-992f-ebaf06eabfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:12.975029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9a542-659e-4e5a-992f-ebaf06eabfc0 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:12.975029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9a542-659e-4e5a-992f-ebaf06eabfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:12.975029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9a542-659e-4e5a-992f-ebaf06eabfc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:12.975860 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9a542-659e-4e5a-992f-ebaf06eabfc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:12.977816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9a542-659e-4e5a-992f-ebaf06eabfc0 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:13.000332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8c5303b9-544d-4f72-83c9-d6a6c0e96757 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dfe380cce1e4dc8b016422e4f40f547', 'request_id': 'req-8c5303b9-544d-4f72-83c9-d6a6c0e96757'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:13.007355 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1320/2682] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:37:12 2022] POST /identity/v3/projects => generated 357 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:37:13.009056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8c5303b9-544d-4f72-83c9-d6a6c0e96757 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dfe380cce1e4dc8b016422e4f40f547', 'request_id': 'req-8c5303b9-544d-4f72-83c9-d6a6c0e96757'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:13.010432 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9a542-659e-4e5a-992f-ebaf06eabfc0 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:13.018752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8c5303b9-544d-4f72-83c9-d6a6c0e96757 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dfe380cce1e4dc8b016422e4f40f547', 'request_id': 'req-8c5303b9-544d-4f72-83c9-d6a6c0e96757'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:13.034834 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8c5303b9-544d-4f72-83c9-d6a6c0e96757 admin admin] The token cache is being invalidate because project 9dfe380cce1e4dc8b016422e4f40f547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:37:13.035226 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8c5303b9-544d-4f72-83c9-d6a6c0e96757 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:13.036484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-8c5303b9-544d-4f72-83c9-d6a6c0e96757 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:13.043128 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1363/2683] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:37:12 2022] DELETE /identity/v3/projects/9dfe380cce1e4dc8b016422e4f40f547 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:13.044814 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d4f8e-bb16-4976-a58e-34b72b60d684 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.045101 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d4f8e-bb16-4976-a58e-34b72b60d684 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.045536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d4f8e-bb16-4976-a58e-34b72b60d684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.067201 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-4e0d4f8e-bb16-4976-a58e-34b72b60d684 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:13.068595 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4e0d4f8e-bb16-4976-a58e-34b72b60d684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:13.096633 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9a542-659e-4e5a-992f-ebaf06eabfc0 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:13.103055 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74361e9e-02d0-4a57-a843-705052f76cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:13.104100 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74361e9e-02d0-4a57-a843-705052f76cdd admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.104224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74361e9e-02d0-4a57-a843-705052f76cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.104425 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74361e9e-02d0-4a57-a843-705052f76cdd admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.109068 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74361e9e-02d0-4a57-a843-705052f76cdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:13.110729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74361e9e-02d0-4a57-a843-705052f76cdd admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:13.137302 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1321/2684] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:37:13 2022] POST /identity/v3/users => generated 285 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:13.137458 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1364/2685] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:13.141227 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74361e9e-02d0-4a57-a843-705052f76cdd admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:13.142979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-74361e9e-02d0-4a57-a843-705052f76cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:13.147118 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490152d-39e5-4a36-94c0-a248a12dfef1 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.147399 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490152d-39e5-4a36-94c0-a248a12dfef1 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.147667 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490152d-39e5-4a36-94c0-a248a12dfef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.165674 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74361e9e-02d0-4a57-a843-705052f76cdd admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:13.172376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33806c36-0f98-419a-8ac2-82e0565fd255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:13.172737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-d490152d-39e5-4a36-94c0-a248a12dfef1 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:13.173042 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33806c36-0f98-419a-8ac2-82e0565fd255 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.173321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33806c36-0f98-419a-8ac2-82e0565fd255 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.173595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33806c36-0f98-419a-8ac2-82e0565fd255 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.173993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-d490152d-39e5-4a36-94c0-a248a12dfef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:13.174320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33806c36-0f98-419a-8ac2-82e0565fd255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:13.176118 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33806c36-0f98-419a-8ac2-82e0565fd255 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:13.183397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1365/2686] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:13.187917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490152d-39e5-4a36-94c0-a248a12dfef1 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:13.191904 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1322/2687] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:13.193509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33806c36-0f98-419a-8ac2-82e0565fd255 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:13.198271 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490152d-39e5-4a36-94c0-a248a12dfef1 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:13.205882 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33806c36-0f98-419a-8ac2-82e0565fd255 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:13.206755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c14e1-3e3d-4975-b43f-9470459386ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:13.207494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c14e1-3e3d-4975-b43f-9470459386ca admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.207815 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c14e1-3e3d-4975-b43f-9470459386ca admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.208087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c14e1-3e3d-4975-b43f-9470459386ca admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.208807 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c14e1-3e3d-4975-b43f-9470459386ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:37:13.209126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c14e1-3e3d-4975-b43f-9470459386ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:13.210724 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c14e1-3e3d-4975-b43f-9470459386ca admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:13.211544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ab66a-7bed-44a8-a589-448d4138d7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:13.212444 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ab66a-7bed-44a8-a589-448d4138d7f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.212729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ab66a-7bed-44a8-a589-448d4138d7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.213008 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ab66a-7bed-44a8-a589-448d4138d7f5 admin admin] PATH_INFO: `/v3/projects/446a926a89144755ad04d4428a9acc74/users/1fbf7d3be2014b00a9a51e6b411f0b24/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.226260 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1366/2688] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:37:13.227511 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c14e1-3e3d-4975-b43f-9470459386ca admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:13.238028 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ab66a-7bed-44a8-a589-448d4138d7f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=446a926a89144755ad04d4428a9acc74, user_id=1fbf7d3be2014b00a9a51e6b411f0b24, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:13.240155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c14e1-3e3d-4975-b43f-9470459386ca admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:13.241244 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ab66a-7bed-44a8-a589-448d4138d7f5 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:13.246180 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6e281-cf8e-4dd2-8648-ed7482bccb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:13.246886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6e281-cf8e-4dd2-8648-ed7482bccb11 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.247190 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6e281-cf8e-4dd2-8648-ed7482bccb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.247508 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6e281-cf8e-4dd2-8648-ed7482bccb11 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.248166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6e281-cf8e-4dd2-8648-ed7482bccb11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:13.250034 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6e281-cf8e-4dd2-8648-ed7482bccb11 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:13.256611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1323/2689] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:37:13 2022] PUT /identity/v3/projects/446a926a89144755ad04d4428a9acc74/users/1fbf7d3be2014b00a9a51e6b411f0b24/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:13.260573 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ab66a-7bed-44a8-a589-448d4138d7f5 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:13.291751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1367/2690] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:37:13 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:37:13.295667 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6e281-cf8e-4dd2-8648-ed7482bccb11 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:13.336218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ab66a-7bed-44a8-a589-448d4138d7f5 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:13.372535 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6e281-cf8e-4dd2-8648-ed7482bccb11 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:13.387551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf697f8-3f35-4fff-b4a4-8f7a52255fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:13.390800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf697f8-3f35-4fff-b4a4-8f7a52255fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.391484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf697f8-3f35-4fff-b4a4-8f7a52255fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.391484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf697f8-3f35-4fff-b4a4-8f7a52255fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.392233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf697f8-3f35-4fff-b4a4-8f7a52255fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:13.394749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf697f8-3f35-4fff-b4a4-8f7a52255fc1 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:13.425177 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1368/2691] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:37:13 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:13.427771 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c43e1-1e26-4691-9769-afaa79e8cd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:13.428458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c43e1-1e26-4691-9769-afaa79e8cd1c admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.428743 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c43e1-1e26-4691-9769-afaa79e8cd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.429018 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c43e1-1e26-4691-9769-afaa79e8cd1c admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.429752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c43e1-1e26-4691-9769-afaa79e8cd1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:13.431363 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c43e1-1e26-4691-9769-afaa79e8cd1c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:13.432384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf697f8-3f35-4fff-b4a4-8f7a52255fc1 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:13.445545 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1324/2692] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:37:13.445860 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf697f8-3f35-4fff-b4a4-8f7a52255fc1 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:13.446767 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c43e1-1e26-4691-9769-afaa79e8cd1c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:13.454512 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8377e5d-cc95-4ee2-b4d5-98bf9dc7628f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:13.454818 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8377e5d-cc95-4ee2-b4d5-98bf9dc7628f admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.455661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8377e5d-cc95-4ee2-b4d5-98bf9dc7628f admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.455661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8377e5d-cc95-4ee2-b4d5-98bf9dc7628f admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.456089 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8377e5d-cc95-4ee2-b4d5-98bf9dc7628f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:13.457714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8377e5d-cc95-4ee2-b4d5-98bf9dc7628f admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:13.459514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c43e1-1e26-4691-9769-afaa79e8cd1c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:13.464918 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b572657-22ef-4ed2-b247-6944e1e7e5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:13.465921 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b572657-22ef-4ed2-b247-6944e1e7e5b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.466296 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b572657-22ef-4ed2-b247-6944e1e7e5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.466665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b572657-22ef-4ed2-b247-6944e1e7e5b6 admin admin] PATH_INFO: `/v3/projects/446a926a89144755ad04d4428a9acc74/users/1fbf7d3be2014b00a9a51e6b411f0b24/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.470142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b572657-22ef-4ed2-b247-6944e1e7e5b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=446a926a89144755ad04d4428a9acc74, user_id=1fbf7d3be2014b00a9a51e6b411f0b24, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:13.471274 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1369/2693] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:13.475548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b572657-22ef-4ed2-b247-6944e1e7e5b6 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:13.476217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8377e5d-cc95-4ee2-b4d5-98bf9dc7628f admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:13.491220 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1325/2694] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:37:13 2022] PUT /identity/v3/projects/446a926a89144755ad04d4428a9acc74/users/1fbf7d3be2014b00a9a51e6b411f0b24/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:13.492832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8377e5d-cc95-4ee2-b4d5-98bf9dc7628f admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:13.498831 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f341238-7475-41ba-bef8-f5517e512714 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.499197 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f341238-7475-41ba-bef8-f5517e512714 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.499728 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f341238-7475-41ba-bef8-f5517e512714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.536800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-9f341238-7475-41ba-bef8-f5517e512714 None None] MFA Rules not processed for user `1fbf7d3be2014b00a9a51e6b411f0b24`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:13.537961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9f341238-7475-41ba-bef8-f5517e512714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:13.570140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d6dc0-3e5e-4bb0-a177-8f0cecc2b8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:13.570826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d6dc0-3e5e-4bb0-a177-8f0cecc2b8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.571224 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d6dc0-3e5e-4bb0-a177-8f0cecc2b8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.571409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d6dc0-3e5e-4bb0-a177-8f0cecc2b8c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.572504 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d6dc0-3e5e-4bb0-a177-8f0cecc2b8c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:13.573766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d6dc0-3e5e-4bb0-a177-8f0cecc2b8c0 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:13.585594 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1370/2695] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:13.591526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d6dc0-3e5e-4bb0-a177-8f0cecc2b8c0 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:13.602299 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d6dc0-3e5e-4bb0-a177-8f0cecc2b8c0 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:13.607855 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8815239-5a8b-4b41-8cfd-2a508cb5328a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:13.609277 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8815239-5a8b-4b41-8cfd-2a508cb5328a admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.609560 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8815239-5a8b-4b41-8cfd-2a508cb5328a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.609841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8815239-5a8b-4b41-8cfd-2a508cb5328a admin admin] PATH_INFO: `/v3/projects/05197c0440624fce9090b97e6826d516/users/cdb5118d48d0412f8536c47b9dffb88c/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.623646 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1326/2696] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:13 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:37:13.628552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8815239-5a8b-4b41-8cfd-2a508cb5328a admin admin] RBAC: Authorizing `identity:create_grant(project_id=05197c0440624fce9090b97e6826d516, user_id=cdb5118d48d0412f8536c47b9dffb88c, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:13.631427 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8815239-5a8b-4b41-8cfd-2a508cb5328a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:13.638651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f341238-7475-41ba-bef8-f5517e512714 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:13.638651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9f341238-7475-41ba-bef8-f5517e512714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:13.649893 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1371/2697] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:37:13 2022] PUT /identity/v3/projects/05197c0440624fce9090b97e6826d516/users/cdb5118d48d0412f8536c47b9dffb88c/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:13.655003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed0160-2a76-4854-842b-73f15b9ba5d9 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.655419 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed0160-2a76-4854-842b-73f15b9ba5d9 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.655772 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed0160-2a76-4854-842b-73f15b9ba5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.676031 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-88ed0160-2a76-4854-842b-73f15b9ba5d9 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:13.677990 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-88ed0160-2a76-4854-842b-73f15b9ba5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:13.745156 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f341238-7475-41ba-bef8-f5517e512714 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:13.750605 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1372/2698] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:37:13.759911 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed0160-2a76-4854-842b-73f15b9ba5d9 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:13.761380 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-88ed0160-2a76-4854-842b-73f15b9ba5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:13.799643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd79ce-2ea0-467e-8f8d-16c8f160df89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:13.799643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd79ce-2ea0-467e-8f8d-16c8f160df89 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.799643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd79ce-2ea0-467e-8f8d-16c8f160df89 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.799643 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd79ce-2ea0-467e-8f8d-16c8f160df89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.829506 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed0160-2a76-4854-842b-73f15b9ba5d9 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:13.833770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c877a93-c29a-4d2b-b596-903cf83704e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:13.834433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c877a93-c29a-4d2b-b596-903cf83704e2 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.834762 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c877a93-c29a-4d2b-b596-903cf83704e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.835045 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c877a93-c29a-4d2b-b596-903cf83704e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.845071 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c877a93-c29a-4d2b-b596-903cf83704e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:13.846481 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c877a93-c29a-4d2b-b596-903cf83704e2 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:13.858076 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1373/2699] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:37:13.866938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd79ce-2ea0-467e-8f8d-16c8f160df89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:13.868468 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd79ce-2ea0-467e-8f8d-16c8f160df89 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:13.878673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1327/2700] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:13 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:37:13.956875 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c877a93-c29a-4d2b-b596-903cf83704e2 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:13.975171 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c877a93-c29a-4d2b-b596-903cf83704e2 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:13.978929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f48c64-ffbe-4d70-bbf7-5612c187e2f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:13.979611 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48c64-ffbe-4d70-bbf7-5612c187e2f8 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:13.979895 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48c64-ffbe-4d70-bbf7-5612c187e2f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:13.980174 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f48c64-ffbe-4d70-bbf7-5612c187e2f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:13.981999 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-74f48c64-ffbe-4d70-bbf7-5612c187e2f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:14.045142 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f48c64-ffbe-4d70-bbf7-5612c187e2f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:14.046582 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f48c64-ffbe-4d70-bbf7-5612c187e2f8 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:14.059685 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1374/2701] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:13 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:14.828116 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae86d6e-ccea-42f6-8cea-2fbab4f34bfe None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:14.828658 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae86d6e-ccea-42f6-8cea-2fbab4f34bfe None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:14.828760 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae86d6e-ccea-42f6-8cea-2fbab4f34bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:14.858786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-9ae86d6e-ccea-42f6-8cea-2fbab4f34bfe None None] MFA Rules not processed for user `cdb5118d48d0412f8536c47b9dffb88c`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:14.860016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9ae86d6e-ccea-42f6-8cea-2fbab4f34bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:14.926078 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1328/2702] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:14.937432 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f48c64-ffbe-4d70-bbf7-5612c187e2f8 service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:14.938887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-74f48c64-ffbe-4d70-bbf7-5612c187e2f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:14.949769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f48c64-ffbe-4d70-bbf7-5612c187e2f8 service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:14.953605 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f93470-7d22-4b24-8753-775855d1afc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:14.954282 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f93470-7d22-4b24-8753-775855d1afc1 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:14.954562 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f93470-7d22-4b24-8753-775855d1afc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:14.954835 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f93470-7d22-4b24-8753-775855d1afc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:14.965375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f93470-7d22-4b24-8753-775855d1afc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:14.966863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f93470-7d22-4b24-8753-775855d1afc1 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:14.977212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1375/2703] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:16.327828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8452b-8b99-483c-8510-a941794a1f5b None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:16.328399 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8452b-8b99-483c-8510-a941794a1f5b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:16.328399 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8452b-8b99-483c-8510-a941794a1f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:16.345460 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-7bc8452b-8b99-483c-8510-a941794a1f5b None None] MFA Rules not processed for user `cdb5118d48d0412f8536c47b9dffb88c`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:16.346644 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7bc8452b-8b99-483c-8510-a941794a1f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:16.355087 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1329/2704] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:16.592599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f93470-7d22-4b24-8753-775855d1afc1 service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:16.609044 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f93470-7d22-4b24-8753-775855d1afc1 service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:16.613291 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908adae-54a9-44ad-80ef-5401df996fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:16.614619 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908adae-54a9-44ad-80ef-5401df996fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:16.614906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908adae-54a9-44ad-80ef-5401df996fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:16.615191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908adae-54a9-44ad-80ef-5401df996fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:16.616971 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-d908adae-54a9-44ad-80ef-5401df996fa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:16.632288 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908adae-54a9-44ad-80ef-5401df996fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:16.633720 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908adae-54a9-44ad-80ef-5401df996fa0 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:16.642552 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1376/2705] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:21.982974 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e66280-a003-44f2-b34d-142972a235b4 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:21.983619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e66280-a003-44f2-b34d-142972a235b4 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:21.983619 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e66280-a003-44f2-b34d-142972a235b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:22.001930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-31e66280-a003-44f2-b34d-142972a235b4 None None] MFA Rules not processed for user `d1934a2e2edb4b8a93baac501c9c5c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:22.003445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-31e66280-a003-44f2-b34d-142972a235b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:22.066596 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1330/2706] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:28.996462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908adae-54a9-44ad-80ef-5401df996fa0 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:29.008494 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908adae-54a9-44ad-80ef-5401df996fa0 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:29.013002 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc1001-0738-4a5f-834d-015395b77d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:29.013887 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc1001-0738-4a5f-834d-015395b77d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:29.014165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc1001-0738-4a5f-834d-015395b77d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:29.014626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc1001-0738-4a5f-834d-015395b77d08 admin admin] PATH_INFO: `/v3/users/6cdb190d146a48f2891b2a7967ab59c1` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:29.015903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc1001-0738-4a5f-834d-015395b77d08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cdb190d146a48f2891b2a7967ab59c1)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:29.018022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc1001-0738-4a5f-834d-015395b77d08 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:29.068332 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-67dc1001-0738-4a5f-834d-015395b77d08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cdb190d146a48f2891b2a7967ab59c1'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.069023 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-67dc1001-0738-4a5f-834d-015395b77d08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cdb190d146a48f2891b2a7967ab59c1'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.077654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-67dc1001-0738-4a5f-834d-015395b77d08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cdb190d146a48f2891b2a7967ab59c1'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.085929 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-67dc1001-0738-4a5f-834d-015395b77d08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cdb190d146a48f2891b2a7967ab59c1'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.086745 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-67dc1001-0738-4a5f-834d-015395b77d08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cdb190d146a48f2891b2a7967ab59c1'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.104557 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1377/2707] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:37:28 2022] DELETE /identity/v3/users/6cdb190d146a48f2891b2a7967ab59c1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:29.109140 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442f600-d91b-438c-9c04-5c8fc5d07a88 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:29.109650 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442f600-d91b-438c-9c04-5c8fc5d07a88 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:29.109753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442f600-d91b-438c-9c04-5c8fc5d07a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:29.126847 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-1442f600-d91b-438c-9c04-5c8fc5d07a88 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:29.128071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1442f600-d91b-438c-9c04-5c8fc5d07a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:29.193295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1331/2708] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:29.203120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc1001-0738-4a5f-834d-015395b77d08 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:29.205078 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-67dc1001-0738-4a5f-834d-015395b77d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:29.279146 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc1001-0738-4a5f-834d-015395b77d08 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:29.282908 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf09099-b4ba-4259-a0c8-0a96214b7818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:29.283601 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09099-b4ba-4259-a0c8-0a96214b7818 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:29.283882 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09099-b4ba-4259-a0c8-0a96214b7818 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:29.284154 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09099-b4ba-4259-a0c8-0a96214b7818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:29.293630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf09099-b4ba-4259-a0c8-0a96214b7818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:29.295311 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf09099-b4ba-4259-a0c8-0a96214b7818 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:29.305347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1378/2709] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:29.590328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442f600-d91b-438c-9c04-5c8fc5d07a88 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:29.593191 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1442f600-d91b-438c-9c04-5c8fc5d07a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:29.612307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442f600-d91b-438c-9c04-5c8fc5d07a88 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:29.617979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:29.618900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:29.619406 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:29.619940 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] PATH_INFO: `/v3/projects/2f67c315056b44ad98b6143428d77ebd` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:29.621125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f67c315056b44ad98b6143428d77ebd)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:29.623275 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:29.676837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f67c315056b44ad98b6143428d77ebd', 'request_id': 'req-74731eb7-0c1d-404d-8de0-c70cd97a237d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.682744 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f67c315056b44ad98b6143428d77ebd', 'request_id': 'req-74731eb7-0c1d-404d-8de0-c70cd97a237d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.694150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f67c315056b44ad98b6143428d77ebd', 'request_id': 'req-74731eb7-0c1d-404d-8de0-c70cd97a237d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.708074 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] The token cache is being invalidate because project 2f67c315056b44ad98b6143428d77ebd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:37:29.708370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.709080 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.710660 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1332/2710] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:37:29 2022] DELETE /identity/v3/projects/2f67c315056b44ad98b6143428d77ebd => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:29.715003 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf09099-b4ba-4259-a0c8-0a96214b7818 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:29.727403 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf09099-b4ba-4259-a0c8-0a96214b7818 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:29.792563 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274db36c-cb68-431c-b64f-e8a05d13d1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:29.793487 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274db36c-cb68-431c-b64f-e8a05d13d1c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:29.793774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274db36c-cb68-431c-b64f-e8a05d13d1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:29.794075 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274db36c-cb68-431c-b64f-e8a05d13d1c3 admin admin] PATH_INFO: `/v3/users/1fbf7d3be2014b00a9a51e6b411f0b24` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:29.795315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274db36c-cb68-431c-b64f-e8a05d13d1c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fbf7d3be2014b00a9a51e6b411f0b24)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:29.797215 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274db36c-cb68-431c-b64f-e8a05d13d1c3 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:29.847125 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-274db36c-cb68-431c-b64f-e8a05d13d1c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fbf7d3be2014b00a9a51e6b411f0b24'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.848992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-274db36c-cb68-431c-b64f-e8a05d13d1c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fbf7d3be2014b00a9a51e6b411f0b24'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.861315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-274db36c-cb68-431c-b64f-e8a05d13d1c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fbf7d3be2014b00a9a51e6b411f0b24'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.868903 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-274db36c-cb68-431c-b64f-e8a05d13d1c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fbf7d3be2014b00a9a51e6b411f0b24'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.869895 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-274db36c-cb68-431c-b64f-e8a05d13d1c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fbf7d3be2014b00a9a51e6b411f0b24'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:29.890035 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1379/2711] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:37:29 2022] DELETE /identity/v3/users/1fbf7d3be2014b00a9a51e6b411f0b24 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:30.262560 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:30.264081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:30.279040 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74731eb7-0c1d-404d-8de0-c70cd97a237d admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:30.337306 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:30.338207 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:30.338492 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:30.338777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5 admin admin] PATH_INFO: `/v3/projects/446a926a89144755ad04d4428a9acc74` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:30.340019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=446a926a89144755ad04d4428a9acc74)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:30.341929 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:30.388452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '446a926a89144755ad04d4428a9acc74', 'request_id': 'req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:30.393843 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '446a926a89144755ad04d4428a9acc74', 'request_id': 'req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:30.401176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '446a926a89144755ad04d4428a9acc74', 'request_id': 'req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:30.412289 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5 admin admin] The token cache is being invalidate because project 446a926a89144755ad04d4428a9acc74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:37:30.412577 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:30.413212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-3d059b79-6cb8-4e91-b1d0-95f5fbe320d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:30.414944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1333/2712] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:37:30 2022] DELETE /identity/v3/projects/446a926a89144755ad04d4428a9acc74 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:30.419709 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9a7f2-b66e-4b35-aa1c-7e4783458be9 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:30.420648 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9a7f2-b66e-4b35-aa1c-7e4783458be9 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:30.420930 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9a7f2-b66e-4b35-aa1c-7e4783458be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:30.438996 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-bdb9a7f2-b66e-4b35-aa1c-7e4783458be9 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:30.440221 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-bdb9a7f2-b66e-4b35-aa1c-7e4783458be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:30.498059 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1380/2713] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:30.506470 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08de03-be5c-40fc-93d0-1312ec0f1f1b None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:30.506763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08de03-be5c-40fc-93d0-1312ec0f1f1b None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:30.507044 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08de03-be5c-40fc-93d0-1312ec0f1f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:30.524691 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-aa08de03-be5c-40fc-93d0-1312ec0f1f1b None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:30.525865 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-aa08de03-be5c-40fc-93d0-1312ec0f1f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:30.533802 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1334/2714] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:30.538183 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9a7f2-b66e-4b35-aa1c-7e4783458be9 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:30.547749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9a7f2-b66e-4b35-aa1c-7e4783458be9 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:30.552845 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4bcaa-1016-41ca-964c-eb0ab72af79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:30.553495 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4bcaa-1016-41ca-964c-eb0ab72af79b admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:30.553784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4bcaa-1016-41ca-964c-eb0ab72af79b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:30.554067 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4bcaa-1016-41ca-964c-eb0ab72af79b admin admin] PATH_INFO: `/v3/domains` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:30.554781 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4bcaa-1016-41ca-964c-eb0ab72af79b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:37:30.555088 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4bcaa-1016-41ca-964c-eb0ab72af79b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:30.556693 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4bcaa-1016-41ca-964c-eb0ab72af79b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:30.568262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1381/2715] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:37:30.572509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08de03-be5c-40fc-93d0-1312ec0f1f1b None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:30.581681 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08de03-be5c-40fc-93d0-1312ec0f1f1b None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:30.586351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26ff4f-37c1-4db1-b6a9-3bd8bec8a66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:30.587001 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26ff4f-37c1-4db1-b6a9-3bd8bec8a66d admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:30.587320 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26ff4f-37c1-4db1-b6a9-3bd8bec8a66d admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:30.587600 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26ff4f-37c1-4db1-b6a9-3bd8bec8a66d admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:30.588288 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26ff4f-37c1-4db1-b6a9-3bd8bec8a66d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:30.590161 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26ff4f-37c1-4db1-b6a9-3bd8bec8a66d admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:30.615423 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1335/2716] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:37:30 2022] POST /identity/v3/projects => generated 363 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:30.619388 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4bcaa-1016-41ca-964c-eb0ab72af79b admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:30.630122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4bcaa-1016-41ca-964c-eb0ab72af79b admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:30.686820 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d232c-fbc5-4742-92af-9726664a1cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:30.687526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d232c-fbc5-4742-92af-9726664a1cdc admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:30.687812 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d232c-fbc5-4742-92af-9726664a1cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:30.688088 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d232c-fbc5-4742-92af-9726664a1cdc admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:30.688770 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d232c-fbc5-4742-92af-9726664a1cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:30.690663 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d232c-fbc5-4742-92af-9726664a1cdc admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:30.712395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1382/2717] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:37:30 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:30.716568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26ff4f-37c1-4db1-b6a9-3bd8bec8a66d admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:30.727508 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26ff4f-37c1-4db1-b6a9-3bd8bec8a66d admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:30.732069 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842906ad-0ff7-47d2-b0e4-8e7e6f7b17a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:30.732714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842906ad-0ff7-47d2-b0e4-8e7e6f7b17a8 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:30.732996 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842906ad-0ff7-47d2-b0e4-8e7e6f7b17a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:30.733287 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842906ad-0ff7-47d2-b0e4-8e7e6f7b17a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:30.733979 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842906ad-0ff7-47d2-b0e4-8e7e6f7b17a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:30.735591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842906ad-0ff7-47d2-b0e4-8e7e6f7b17a8 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:30.746423 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1336/2718] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:30.750222 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d232c-fbc5-4742-92af-9726664a1cdc admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:30.759915 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d232c-fbc5-4742-92af-9726664a1cdc admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:30.764443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0884533-f44e-47d1-a333-ba77fab9f2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:30.765087 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884533-f44e-47d1-a333-ba77fab9f2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:30.765370 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884533-f44e-47d1-a333-ba77fab9f2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:30.765645 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884533-f44e-47d1-a333-ba77fab9f2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:30.766355 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0884533-f44e-47d1-a333-ba77fab9f2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:30.767963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0884533-f44e-47d1-a333-ba77fab9f2a5 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:30.777373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1383/2719] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:30.781156 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842906ad-0ff7-47d2-b0e4-8e7e6f7b17a8 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:30.791309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842906ad-0ff7-47d2-b0e4-8e7e6f7b17a8 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:30.795816 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be0ea5-f916-4138-ab69-57d82d32b9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:30.796678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be0ea5-f916-4138-ab69-57d82d32b9d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:30.796963 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be0ea5-f916-4138-ab69-57d82d32b9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:30.797238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be0ea5-f916-4138-ab69-57d82d32b9d3 admin admin] PATH_INFO: `/v3/projects/a374fa3c53ef413ab1ea7496d976cb4f/users/c431209b96654ffb8e958f5d33e5a269/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:30.811779 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be0ea5-f916-4138-ab69-57d82d32b9d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a374fa3c53ef413ab1ea7496d976cb4f, user_id=c431209b96654ffb8e958f5d33e5a269, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:30.814395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be0ea5-f916-4138-ab69-57d82d32b9d3 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:30.827928 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1337/2720] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:37:30 2022] PUT /identity/v3/projects/a374fa3c53ef413ab1ea7496d976cb4f/users/c431209b96654ffb8e958f5d33e5a269/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:30.838108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31014eda-bbce-4a63-9cee-92aa7a6dd4e4 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:30.838398 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31014eda-bbce-4a63-9cee-92aa7a6dd4e4 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:30.838677 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31014eda-bbce-4a63-9cee-92aa7a6dd4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:30.865406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-31014eda-bbce-4a63-9cee-92aa7a6dd4e4 None None] MFA Rules not processed for user `c431209b96654ffb8e958f5d33e5a269`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:30.866585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-31014eda-bbce-4a63-9cee-92aa7a6dd4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:30.929233 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1384/2721] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:30 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:30.933773 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be0ea5-f916-4138-ab69-57d82d32b9d3 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:30.992427 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be0ea5-f916-4138-ab69-57d82d32b9d3 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:30.997334 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb9efc-e7bc-45c4-8149-de08d66c825c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:30.997992 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb9efc-e7bc-45c4-8149-de08d66c825c admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:30.998273 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb9efc-e7bc-45c4-8149-de08d66c825c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:30.998548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb9efc-e7bc-45c4-8149-de08d66c825c admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:30.999256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb9efc-e7bc-45c4-8149-de08d66c825c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:31.001130 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb9efc-e7bc-45c4-8149-de08d66c825c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:31.025034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1338/2722] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:37:30 2022] POST /identity/v3/projects => generated 363 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:31.029181 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31014eda-bbce-4a63-9cee-92aa7a6dd4e4 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:31.040064 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31014eda-bbce-4a63-9cee-92aa7a6dd4e4 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:31.094597 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3505ed5-7b88-48f3-b05d-1dab6841d420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:31.095291 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3505ed5-7b88-48f3-b05d-1dab6841d420 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:31.095574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3505ed5-7b88-48f3-b05d-1dab6841d420 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:31.095851 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3505ed5-7b88-48f3-b05d-1dab6841d420 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:31.096512 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3505ed5-7b88-48f3-b05d-1dab6841d420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:31.099061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3505ed5-7b88-48f3-b05d-1dab6841d420 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:31.118306 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1385/2723] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:37:31 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:31.122683 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb9efc-e7bc-45c4-8149-de08d66c825c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:31.132753 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb9efc-e7bc-45c4-8149-de08d66c825c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:31.137343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3978b-0b5d-4ce9-b06f-c0da44639217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:31.137998 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3978b-0b5d-4ce9-b06f-c0da44639217 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:31.138284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3978b-0b5d-4ce9-b06f-c0da44639217 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:31.138562 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3978b-0b5d-4ce9-b06f-c0da44639217 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:31.139312 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3978b-0b5d-4ce9-b06f-c0da44639217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:31.140884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3978b-0b5d-4ce9-b06f-c0da44639217 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:31.151819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1339/2724] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:31.155530 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3505ed5-7b88-48f3-b05d-1dab6841d420 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:31.165637 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3505ed5-7b88-48f3-b05d-1dab6841d420 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:31.170227 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d88078-5329-405b-bfa7-dbdce8cfb721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:31.171114 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d88078-5329-405b-bfa7-dbdce8cfb721 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:31.171430 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d88078-5329-405b-bfa7-dbdce8cfb721 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:31.171703 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d88078-5329-405b-bfa7-dbdce8cfb721 admin admin] PATH_INFO: `/v3/projects/3bd2fe85adc04561829b357a5344ba0e/users/fa3168e1f42948bca65c0ab592ddee1c/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:31.185889 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d88078-5329-405b-bfa7-dbdce8cfb721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd2fe85adc04561829b357a5344ba0e, user_id=fa3168e1f42948bca65c0ab592ddee1c, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:31.188512 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d88078-5329-405b-bfa7-dbdce8cfb721 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:31.201125 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1386/2725] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:37:31 2022] PUT /identity/v3/projects/3bd2fe85adc04561829b357a5344ba0e/users/fa3168e1f42948bca65c0ab592ddee1c/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:31.205151 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3978b-0b5d-4ce9-b06f-c0da44639217 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:31.215640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3978b-0b5d-4ce9-b06f-c0da44639217 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:31.271571 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c819cb-12c6-4fcb-8f16-b0f12c8f5955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:31.272240 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c819cb-12c6-4fcb-8f16-b0f12c8f5955 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:31.272516 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c819cb-12c6-4fcb-8f16-b0f12c8f5955 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:31.272787 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c819cb-12c6-4fcb-8f16-b0f12c8f5955 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:31.273508 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c819cb-12c6-4fcb-8f16-b0f12c8f5955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:31.275083 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c819cb-12c6-4fcb-8f16-b0f12c8f5955 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:31.284667 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1340/2726] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:31.289410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d88078-5329-405b-bfa7-dbdce8cfb721 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:31.299872 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d88078-5329-405b-bfa7-dbdce8cfb721 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:31.304861 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac70bb-67af-478f-850f-56bd46afb30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:31.305755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac70bb-67af-478f-850f-56bd46afb30a admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:31.306040 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac70bb-67af-478f-850f-56bd46afb30a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:31.306317 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac70bb-67af-478f-850f-56bd46afb30a admin admin] PATH_INFO: `/v3/projects/3bd2fe85adc04561829b357a5344ba0e/users/fa3168e1f42948bca65c0ab592ddee1c/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:31.308536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac70bb-67af-478f-850f-56bd46afb30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd2fe85adc04561829b357a5344ba0e, user_id=fa3168e1f42948bca65c0ab592ddee1c, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:31.311200 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac70bb-67af-478f-850f-56bd46afb30a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:31.322462 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1387/2727] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:37:31 2022] PUT /identity/v3/projects/3bd2fe85adc04561829b357a5344ba0e/users/fa3168e1f42948bca65c0ab592ddee1c/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:31.333848 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138a7bb-50f2-4ddd-84b4-2f748f1f7dd7 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:31.334142 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138a7bb-50f2-4ddd-84b4-2f748f1f7dd7 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:31.334423 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138a7bb-50f2-4ddd-84b4-2f748f1f7dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:31.360082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-7138a7bb-50f2-4ddd-84b4-2f748f1f7dd7 None None] MFA Rules not processed for user `fa3168e1f42948bca65c0ab592ddee1c`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:31.361271 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7138a7bb-50f2-4ddd-84b4-2f748f1f7dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:31.426019 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1341/2728] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:31 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:31.437963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac70bb-67af-478f-850f-56bd46afb30a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:31.439731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-eeac70bb-67af-478f-850f-56bd46afb30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:31.497238 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac70bb-67af-478f-850f-56bd46afb30a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:31.501072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb6c8d-4644-4e8c-b4cb-9bbf6fcf4e54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:31.501788 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb6c8d-4644-4e8c-b4cb-9bbf6fcf4e54 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:31.502134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb6c8d-4644-4e8c-b4cb-9bbf6fcf4e54 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:31.502475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb6c8d-4644-4e8c-b4cb-9bbf6fcf4e54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:31.511910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb6c8d-4644-4e8c-b4cb-9bbf6fcf4e54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:31.513402 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb6c8d-4644-4e8c-b4cb-9bbf6fcf4e54 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:31.522764 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1388/2729] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:31 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:31.598628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138a7bb-50f2-4ddd-84b4-2f748f1f7dd7 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:31.610626 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138a7bb-50f2-4ddd-84b4-2f748f1f7dd7 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:31.614926 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe84c8-43c6-4330-b650-9601e2c02b94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:31.615599 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe84c8-43c6-4330-b650-9601e2c02b94 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:31.615878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe84c8-43c6-4330-b650-9601e2c02b94 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:31.616178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe84c8-43c6-4330-b650-9601e2c02b94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:31.673193 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe84c8-43c6-4330-b650-9601e2c02b94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:31.674612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe84c8-43c6-4330-b650-9601e2c02b94 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:31.684067 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1342/2730] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:31 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:35.792621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be46c0-4f80-4bd9-bfca-49d6f46faaa2 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:35.793084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be46c0-4f80-4bd9-bfca-49d6f46faaa2 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:35.793178 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be46c0-4f80-4bd9-bfca-49d6f46faaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:35.810976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-76be46c0-4f80-4bd9-bfca-49d6f46faaa2 None None] MFA Rules not processed for user `cdb5118d48d0412f8536c47b9dffb88c`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:35.812185 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-76be46c0-4f80-4bd9-bfca-49d6f46faaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:35.863508 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1389/2731] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:43.916805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe84c8-43c6-4330-b650-9601e2c02b94 service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:43.995551 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe84c8-43c6-4330-b650-9601e2c02b94 service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:44.000496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a5d3f-c32f-46ce-bea7-e2fc09f5dc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:44.001375 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a5d3f-c32f-46ce-bea7-e2fc09f5dc7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:44.001661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a5d3f-c32f-46ce-bea7-e2fc09f5dc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:44.001937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a5d3f-c32f-46ce-bea7-e2fc09f5dc7f admin admin] PATH_INFO: `/v3/users/c431209b96654ffb8e958f5d33e5a269` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:44.003477 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a5d3f-c32f-46ce-bea7-e2fc09f5dc7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c431209b96654ffb8e958f5d33e5a269)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:44.005418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a5d3f-c32f-46ce-bea7-e2fc09f5dc7f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:44.062524 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b09a5d3f-c32f-46ce-bea7-e2fc09f5dc7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c431209b96654ffb8e958f5d33e5a269'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:44.074496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b09a5d3f-c32f-46ce-bea7-e2fc09f5dc7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c431209b96654ffb8e958f5d33e5a269'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:44.075246 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b09a5d3f-c32f-46ce-bea7-e2fc09f5dc7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c431209b96654ffb8e958f5d33e5a269'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:44.084892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b09a5d3f-c32f-46ce-bea7-e2fc09f5dc7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c431209b96654ffb8e958f5d33e5a269'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:44.085900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b09a5d3f-c32f-46ce-bea7-e2fc09f5dc7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c431209b96654ffb8e958f5d33e5a269'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:44.105956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1343/2732] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:37:43 2022] DELETE /identity/v3/users/c431209b96654ffb8e958f5d33e5a269 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:44.110506 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb35cc-c631-441d-92b7-33b301be66ca None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:44.111009 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb35cc-c631-441d-92b7-33b301be66ca None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:44.111109 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb35cc-c631-441d-92b7-33b301be66ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:44.133856 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-fbdb35cc-c631-441d-92b7-33b301be66ca None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:44.135059 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-fbdb35cc-c631-441d-92b7-33b301be66ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:44.194122 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1390/2733] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:44.203820 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a5d3f-c32f-46ce-bea7-e2fc09f5dc7f admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:44.205195 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b09a5d3f-c32f-46ce-bea7-e2fc09f5dc7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:44.261546 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a5d3f-c32f-46ce-bea7-e2fc09f5dc7f admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:44.265123 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eca277-91c6-45ae-84c9-9a363004f57c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:44.265764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eca277-91c6-45ae-84c9-9a363004f57c service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:44.266049 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eca277-91c6-45ae-84c9-9a363004f57c service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:44.266328 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eca277-91c6-45ae-84c9-9a363004f57c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:44.275624 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eca277-91c6-45ae-84c9-9a363004f57c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:44.277027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eca277-91c6-45ae-84c9-9a363004f57c service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:44.286758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1344/2734] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:44.714129 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb35cc-c631-441d-92b7-33b301be66ca None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:44.715705 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-fbdb35cc-c631-441d-92b7-33b301be66ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:44.748118 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb35cc-c631-441d-92b7-33b301be66ca None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:44.752787 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:44.753654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:44.753937 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:44.754214 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] PATH_INFO: `/v3/projects/a374fa3c53ef413ab1ea7496d976cb4f` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:44.756094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a374fa3c53ef413ab1ea7496d976cb4f)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:44.757997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:44.819717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a374fa3c53ef413ab1ea7496d976cb4f', 'request_id': 'req-2dbcab48-2658-47f5-8c94-8d524152cb01'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:44.825395 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a374fa3c53ef413ab1ea7496d976cb4f', 'request_id': 'req-2dbcab48-2658-47f5-8c94-8d524152cb01'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:44.833576 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a374fa3c53ef413ab1ea7496d976cb4f', 'request_id': 'req-2dbcab48-2658-47f5-8c94-8d524152cb01'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:44.850236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] The token cache is being invalidate because project a374fa3c53ef413ab1ea7496d976cb4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:37:44.850951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:44.852433 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:44.854957 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1391/2735] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:37:44 2022] DELETE /identity/v3/projects/a374fa3c53ef413ab1ea7496d976cb4f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:44.858968 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eca277-91c6-45ae-84c9-9a363004f57c service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:44.872115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eca277-91c6-45ae-84c9-9a363004f57c service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:44.933358 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10535e07-24f2-44a3-b157-03b56bf7f28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:44.934231 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10535e07-24f2-44a3-b157-03b56bf7f28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:44.934513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10535e07-24f2-44a3-b157-03b56bf7f28d admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:44.934786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10535e07-24f2-44a3-b157-03b56bf7f28d admin admin] PATH_INFO: `/v3/users/fa3168e1f42948bca65c0ab592ddee1c` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:44.936043 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10535e07-24f2-44a3-b157-03b56bf7f28d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa3168e1f42948bca65c0ab592ddee1c)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:44.937921 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10535e07-24f2-44a3-b157-03b56bf7f28d admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:44.991683 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-10535e07-24f2-44a3-b157-03b56bf7f28d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa3168e1f42948bca65c0ab592ddee1c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:45.000559 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-10535e07-24f2-44a3-b157-03b56bf7f28d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa3168e1f42948bca65c0ab592ddee1c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:45.001551 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-10535e07-24f2-44a3-b157-03b56bf7f28d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa3168e1f42948bca65c0ab592ddee1c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:45.011746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-10535e07-24f2-44a3-b157-03b56bf7f28d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa3168e1f42948bca65c0ab592ddee1c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:45.012345 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-10535e07-24f2-44a3-b157-03b56bf7f28d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa3168e1f42948bca65c0ab592ddee1c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:45.030257 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1345/2736] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:37:44 2022] DELETE /identity/v3/users/fa3168e1f42948bca65c0ab592ddee1c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:45.341520 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:45.343061 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:45.362892 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcab48-2658-47f5-8c94-8d524152cb01 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:45.422835 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68814c65-da3c-46b6-b92c-9062f08a3a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:45.424019 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68814c65-da3c-46b6-b92c-9062f08a3a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:45.424304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68814c65-da3c-46b6-b92c-9062f08a3a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:45.424572 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68814c65-da3c-46b6-b92c-9062f08a3a88 admin admin] PATH_INFO: `/v3/projects/3bd2fe85adc04561829b357a5344ba0e` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:45.425794 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68814c65-da3c-46b6-b92c-9062f08a3a88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd2fe85adc04561829b357a5344ba0e)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:45.429502 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68814c65-da3c-46b6-b92c-9062f08a3a88 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:45.480096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-68814c65-da3c-46b6-b92c-9062f08a3a88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd2fe85adc04561829b357a5344ba0e', 'request_id': 'req-68814c65-da3c-46b6-b92c-9062f08a3a88'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:45.485405 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-68814c65-da3c-46b6-b92c-9062f08a3a88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd2fe85adc04561829b357a5344ba0e', 'request_id': 'req-68814c65-da3c-46b6-b92c-9062f08a3a88'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:45.494062 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-68814c65-da3c-46b6-b92c-9062f08a3a88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd2fe85adc04561829b357a5344ba0e', 'request_id': 'req-68814c65-da3c-46b6-b92c-9062f08a3a88'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:45.508114 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-68814c65-da3c-46b6-b92c-9062f08a3a88 admin admin] The token cache is being invalidate because project 3bd2fe85adc04561829b357a5344ba0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:37:45.508386 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-68814c65-da3c-46b6-b92c-9062f08a3a88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:45.509054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-68814c65-da3c-46b6-b92c-9062f08a3a88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:37:45.513958 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1392/2737] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:37:45 2022] DELETE /identity/v3/projects/3bd2fe85adc04561829b357a5344ba0e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:45.515767 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98185b4-ce3b-424f-a81b-08f3aa6bb83a None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:45.516060 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98185b4-ce3b-424f-a81b-08f3aa6bb83a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:45.516339 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98185b4-ce3b-424f-a81b-08f3aa6bb83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:45.535204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-b98185b4-ce3b-424f-a81b-08f3aa6bb83a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:45.536929 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b98185b4-ce3b-424f-a81b-08f3aa6bb83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:45.598065 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1346/2738] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:45.606579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cf1b3-e2cb-4eed-ab46-1c6a6f4b6a48 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:45.606868 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cf1b3-e2cb-4eed-ab46-1c6a6f4b6a48 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:45.607146 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cf1b3-e2cb-4eed-ab46-1c6a6f4b6a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:45.626217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-2d9cf1b3-e2cb-4eed-ab46-1c6a6f4b6a48 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:45.627525 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-2d9cf1b3-e2cb-4eed-ab46-1c6a6f4b6a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:45.636706 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1393/2739] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:37:45.641084 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98185b4-ce3b-424f-a81b-08f3aa6bb83a None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:45.653161 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98185b4-ce3b-424f-a81b-08f3aa6bb83a None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:45.657776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811af18-061b-4135-9460-92b05df5127d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:45.658441 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811af18-061b-4135-9460-92b05df5127d admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:45.658724 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811af18-061b-4135-9460-92b05df5127d admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:45.659050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811af18-061b-4135-9460-92b05df5127d admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:45.659811 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811af18-061b-4135-9460-92b05df5127d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:37:45.660120 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811af18-061b-4135-9460-92b05df5127d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:45.662251 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811af18-061b-4135-9460-92b05df5127d admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:45.676027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1347/2740] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:37:45.680924 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cf1b3-e2cb-4eed-ab46-1c6a6f4b6a48 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:45.691654 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cf1b3-e2cb-4eed-ab46-1c6a6f4b6a48 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:45.697435 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6823bcf-d946-4d52-ad99-a6093c8aa939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:45.698250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6823bcf-d946-4d52-ad99-a6093c8aa939 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:45.698526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6823bcf-d946-4d52-ad99-a6093c8aa939 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:45.698964 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6823bcf-d946-4d52-ad99-a6093c8aa939 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:45.699676 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6823bcf-d946-4d52-ad99-a6093c8aa939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:45.701922 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6823bcf-d946-4d52-ad99-a6093c8aa939 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:45.735102 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1394/2741] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:37:45 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:45.738858 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811af18-061b-4135-9460-92b05df5127d admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:45.751540 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811af18-061b-4135-9460-92b05df5127d admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:45.810896 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eb016-8474-49e6-9f2c-1c988066a4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:45.811700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb016-8474-49e6-9f2c-1c988066a4b0 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:45.811990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb016-8474-49e6-9f2c-1c988066a4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:45.812268 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb016-8474-49e6-9f2c-1c988066a4b0 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:45.812971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eb016-8474-49e6-9f2c-1c988066a4b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:45.815925 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eb016-8474-49e6-9f2c-1c988066a4b0 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:45.839292 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1348/2742] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:37:45 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:37:45.843570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6823bcf-d946-4d52-ad99-a6093c8aa939 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:45.855280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6823bcf-d946-4d52-ad99-a6093c8aa939 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:45.861084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341ab12-8833-46a0-8984-d2e2e1eb9453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:45.861846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341ab12-8833-46a0-8984-d2e2e1eb9453 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:45.862198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341ab12-8833-46a0-8984-d2e2e1eb9453 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:45.862480 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341ab12-8833-46a0-8984-d2e2e1eb9453 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:45.863231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341ab12-8833-46a0-8984-d2e2e1eb9453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:45.864986 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341ab12-8833-46a0-8984-d2e2e1eb9453 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:45.876604 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1395/2743] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:37:45.880390 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eb016-8474-49e6-9f2c-1c988066a4b0 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:45.890273 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eb016-8474-49e6-9f2c-1c988066a4b0 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:45.894877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d03e3-a262-4a99-9003-fdf0b7c2b55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:45.896108 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d03e3-a262-4a99-9003-fdf0b7c2b55b admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:45.896392 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d03e3-a262-4a99-9003-fdf0b7c2b55b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:45.896663 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d03e3-a262-4a99-9003-fdf0b7c2b55b admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:45.897513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d03e3-a262-4a99-9003-fdf0b7c2b55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:45.899132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d03e3-a262-4a99-9003-fdf0b7c2b55b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:45.912164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1349/2744] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:37:45.916052 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341ab12-8833-46a0-8984-d2e2e1eb9453 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:45.925768 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341ab12-8833-46a0-8984-d2e2e1eb9453 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:45.930418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec24dea-f7c3-41d6-99df-0edfc00d12ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:45.931323 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec24dea-f7c3-41d6-99df-0edfc00d12ff admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:45.931608 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec24dea-f7c3-41d6-99df-0edfc00d12ff admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:45.931904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec24dea-f7c3-41d6-99df-0edfc00d12ff admin admin] PATH_INFO: `/v3/projects/dfa792a6828b476a962096e9ae012436/users/cf928a52135e4697bb9f704becde9890/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:45.947290 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec24dea-f7c3-41d6-99df-0edfc00d12ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfa792a6828b476a962096e9ae012436, user_id=cf928a52135e4697bb9f704becde9890, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:45.949673 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec24dea-f7c3-41d6-99df-0edfc00d12ff admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:45.971262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1396/2745] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:37:45 2022] PUT /identity/v3/projects/dfa792a6828b476a962096e9ae012436/users/cf928a52135e4697bb9f704becde9890/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:37:45.982248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486e999-0695-4ca9-910d-ea3290e67ef6 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:45.982799 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486e999-0695-4ca9-910d-ea3290e67ef6 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:45.983042 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486e999-0695-4ca9-910d-ea3290e67ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:46.011334 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-4486e999-0695-4ca9-910d-ea3290e67ef6 None None] MFA Rules not processed for user `cf928a52135e4697bb9f704becde9890`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:46.012668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-4486e999-0695-4ca9-910d-ea3290e67ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:46.074752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1350/2746] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:37:46.079124 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec24dea-f7c3-41d6-99df-0edfc00d12ff admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:46.138243 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec24dea-f7c3-41d6-99df-0edfc00d12ff admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:46.142759 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a04133-8fcc-4d7e-ba59-63c565ca745a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:46.143444 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a04133-8fcc-4d7e-ba59-63c565ca745a admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:46.143727 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a04133-8fcc-4d7e-ba59-63c565ca745a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:46.144004 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a04133-8fcc-4d7e-ba59-63c565ca745a admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:46.144695 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a04133-8fcc-4d7e-ba59-63c565ca745a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:46.147126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a04133-8fcc-4d7e-ba59-63c565ca745a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:46.172241 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1397/2747] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:37:46 2022] POST /identity/v3/projects => generated 375 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:46.176543 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486e999-0695-4ca9-910d-ea3290e67ef6 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:46.187416 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486e999-0695-4ca9-910d-ea3290e67ef6 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:46.243284 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00887496-47d4-45cf-8bca-a42b060eb9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:46.243965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00887496-47d4-45cf-8bca-a42b060eb9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:46.244537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00887496-47d4-45cf-8bca-a42b060eb9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:46.244813 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00887496-47d4-45cf-8bca-a42b060eb9b6 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:46.245496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00887496-47d4-45cf-8bca-a42b060eb9b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:46.247500 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00887496-47d4-45cf-8bca-a42b060eb9b6 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:46.271210 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1351/2748] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:37:46 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:37:46.272222 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a04133-8fcc-4d7e-ba59-63c565ca745a admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:46.282134 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a04133-8fcc-4d7e-ba59-63c565ca745a admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:46.286732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65470424-67aa-45ad-a269-469487cdaccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:46.287418 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65470424-67aa-45ad-a269-469487cdaccc admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:46.287704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65470424-67aa-45ad-a269-469487cdaccc admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:46.287981 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65470424-67aa-45ad-a269-469487cdaccc admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:46.288705 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65470424-67aa-45ad-a269-469487cdaccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:46.290406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65470424-67aa-45ad-a269-469487cdaccc admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:46.302237 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1398/2749] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:46.307023 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00887496-47d4-45cf-8bca-a42b060eb9b6 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:46.317898 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00887496-47d4-45cf-8bca-a42b060eb9b6 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:46.323464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93dd0c6-7e02-4413-828e-5368ca479250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:46.324468 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93dd0c6-7e02-4413-828e-5368ca479250 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:46.324811 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93dd0c6-7e02-4413-828e-5368ca479250 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:46.325167 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93dd0c6-7e02-4413-828e-5368ca479250 admin admin] PATH_INFO: `/v3/projects/9b81db174a08400cab2c6a9637ceebb0/users/0f87a01391d542cf9d02b8adb61bc730/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:46.341567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93dd0c6-7e02-4413-828e-5368ca479250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b81db174a08400cab2c6a9637ceebb0, user_id=0f87a01391d542cf9d02b8adb61bc730, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:46.344399 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93dd0c6-7e02-4413-828e-5368ca479250 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:46.358970 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1352/2750] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:37:46 2022] PUT /identity/v3/projects/9b81db174a08400cab2c6a9637ceebb0/users/0f87a01391d542cf9d02b8adb61bc730/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:46.362831 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65470424-67aa-45ad-a269-469487cdaccc admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:46.373289 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65470424-67aa-45ad-a269-469487cdaccc admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:46.428575 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10bc01d-bf1a-452a-8456-43adbddcbb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:46.429176 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bc01d-bf1a-452a-8456-43adbddcbb6b admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:46.429538 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bc01d-bf1a-452a-8456-43adbddcbb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:46.429821 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bc01d-bf1a-452a-8456-43adbddcbb6b admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:46.430555 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10bc01d-bf1a-452a-8456-43adbddcbb6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:46.432176 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10bc01d-bf1a-452a-8456-43adbddcbb6b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:46.445484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1399/2751] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:37:46.446665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93dd0c6-7e02-4413-828e-5368ca479250 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:46.456932 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93dd0c6-7e02-4413-828e-5368ca479250 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:46.462263 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dc23d-878a-4d2f-864d-c3f76b72d63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:46.463193 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dc23d-878a-4d2f-864d-c3f76b72d63f admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:46.463497 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dc23d-878a-4d2f-864d-c3f76b72d63f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:46.463769 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dc23d-878a-4d2f-864d-c3f76b72d63f admin admin] PATH_INFO: `/v3/projects/9b81db174a08400cab2c6a9637ceebb0/users/0f87a01391d542cf9d02b8adb61bc730/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:46.465996 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dc23d-878a-4d2f-864d-c3f76b72d63f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b81db174a08400cab2c6a9637ceebb0, user_id=0f87a01391d542cf9d02b8adb61bc730, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:46.468668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dc23d-878a-4d2f-864d-c3f76b72d63f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:46.479484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1353/2752] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:37:46 2022] PUT /identity/v3/projects/9b81db174a08400cab2c6a9637ceebb0/users/0f87a01391d542cf9d02b8adb61bc730/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:37:46.492047 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a9b00-69cf-470a-b899-544236c204cf None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:46.492644 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a9b00-69cf-470a-b899-544236c204cf None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:46.492926 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a9b00-69cf-470a-b899-544236c204cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:46.519534 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-9e4a9b00-69cf-470a-b899-544236c204cf None None] MFA Rules not processed for user `0f87a01391d542cf9d02b8adb61bc730`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:37:46.520742 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9e4a9b00-69cf-470a-b899-544236c204cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:46.588574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1400/2753] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:37:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:37:46.599737 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dc23d-878a-4d2f-864d-c3f76b72d63f admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:46.601429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-227dc23d-878a-4d2f-864d-c3f76b72d63f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:46.658505 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dc23d-878a-4d2f-864d-c3f76b72d63f admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:46.662089 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99903574-d991-4c36-bc63-e05a97f98a97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:46.662727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99903574-d991-4c36-bc63-e05a97f98a97 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:46.663006 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99903574-d991-4c36-bc63-e05a97f98a97 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:46.663374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99903574-d991-4c36-bc63-e05a97f98a97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:46.718659 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99903574-d991-4c36-bc63-e05a97f98a97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:46.720031 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99903574-d991-4c36-bc63-e05a97f98a97 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:46.728677 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1354/2754] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:51.318756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a9b00-69cf-470a-b899-544236c204cf None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:51.328896 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a9b00-69cf-470a-b899-544236c204cf None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:51.332626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39a3e7-a101-4914-8621-22cae154b13a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:51.333344 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39a3e7-a101-4914-8621-22cae154b13a service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:51.333629 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39a3e7-a101-4914-8621-22cae154b13a service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:51.333907 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39a3e7-a101-4914-8621-22cae154b13a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:51.343359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39a3e7-a101-4914-8621-22cae154b13a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:51.344938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39a3e7-a101-4914-8621-22cae154b13a service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:51.355585 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1401/2755] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:51 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:37:52.660799 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99903574-d991-4c36-bc63-e05a97f98a97 service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:37:52.662267 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-99903574-d991-4c36-bc63-e05a97f98a97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:52.718775 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99903574-d991-4c36-bc63-e05a97f98a97 service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:37:52.722314 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8325ecc-684d-4345-a725-144744aac974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:37:52.722956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8325ecc-684d-4345-a725-144744aac974 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:37:52.723278 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8325ecc-684d-4345-a725-144744aac974 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:37:52.723558 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8325ecc-684d-4345-a725-144744aac974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:37:52.725289 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-a8325ecc-684d-4345-a725-144744aac974 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:37:52.784161 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8325ecc-684d-4345-a725-144744aac974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:37:52.785993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8325ecc-684d-4345-a725-144744aac974 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:37:52.798906 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1355/2756] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:37:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:38:07.712964 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39a3e7-a101-4914-8621-22cae154b13a service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:07.714421 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-ad39a3e7-a101-4914-8621-22cae154b13a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:07.780690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39a3e7-a101-4914-8621-22cae154b13a service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:07.785617 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9e3d0-8810-4f1d-b7c6-2cc9ffe6410d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f747dcb955fa499a972c46587d3db294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:07.786262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9e3d0-8810-4f1d-b7c6-2cc9ffe6410d service placement] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:07.786541 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9e3d0-8810-4f1d-b7c6-2cc9ffe6410d service placement] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:07.786813 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9e3d0-8810-4f1d-b7c6-2cc9ffe6410d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:07.788568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-bbf9e3d0-8810-4f1d-b7c6-2cc9ffe6410d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:07.799531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9e3d0-8810-4f1d-b7c6-2cc9ffe6410d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:07.801607 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9e3d0-8810-4f1d-b7c6-2cc9ffe6410d service placement] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:07.811217 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1402/2757] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:38:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:38:15.917300 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9e246-c830-45af-a3d4-a0010f84bc13 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:15.917792 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9e246-c830-45af-a3d4-a0010f84bc13 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:15.917971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9e246-c830-45af-a3d4-a0010f84bc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:15.937188 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-69b9e246-c830-45af-a3d4-a0010f84bc13 None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:15.938400 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-69b9e246-c830-45af-a3d4-a0010f84bc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:16.002581 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1356/2758] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:38:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:16.053953 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9e3d0-8810-4f1d-b7c6-2cc9ffe6410d service placement] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:16.055626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-bbf9e3d0-8810-4f1d-b7c6-2cc9ffe6410d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:16.127111 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9e3d0-8810-4f1d-b7c6-2cc9ffe6410d service placement] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:16.131992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6ce72-9493-4842-b751-49b39b9e1701 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:16.132635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6ce72-9493-4842-b751-49b39b9e1701 service swift] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:16.132915 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6ce72-9493-4842-b751-49b39b9e1701 service swift] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:16.133187 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6ce72-9493-4842-b751-49b39b9e1701 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:16.140906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a6ce72-9493-4842-b751-49b39b9e1701 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:16.142442 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a6ce72-9493-4842-b751-49b39b9e1701 service swift] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:16.150195 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1403/2759] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:38:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 09 18:38:21.435648 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9e246-c830-45af-a3d4-a0010f84bc13 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:21.508409 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9e246-c830-45af-a3d4-a0010f84bc13 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:21.512133 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a849f6e-4fc9-4def-8d77-0635ae6f3bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:21.513054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a849f6e-4fc9-4def-8d77-0635ae6f3bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:21.513343 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a849f6e-4fc9-4def-8d77-0635ae6f3bab admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:21.513625 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a849f6e-4fc9-4def-8d77-0635ae6f3bab admin admin] PATH_INFO: `/v3/users/cf928a52135e4697bb9f704becde9890` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:21.515017 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a849f6e-4fc9-4def-8d77-0635ae6f3bab admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf928a52135e4697bb9f704becde9890)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:21.516961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a849f6e-4fc9-4def-8d77-0635ae6f3bab admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:21.591213 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7a849f6e-4fc9-4def-8d77-0635ae6f3bab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf928a52135e4697bb9f704becde9890'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:21.598819 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7a849f6e-4fc9-4def-8d77-0635ae6f3bab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf928a52135e4697bb9f704becde9890'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:21.611943 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7a849f6e-4fc9-4def-8d77-0635ae6f3bab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf928a52135e4697bb9f704becde9890'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:21.655878 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7a849f6e-4fc9-4def-8d77-0635ae6f3bab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf928a52135e4697bb9f704becde9890'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:21.656662 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7a849f6e-4fc9-4def-8d77-0635ae6f3bab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf928a52135e4697bb9f704becde9890'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:21.683354 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1357/2760] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:38:21 2022] DELETE /identity/v3/users/cf928a52135e4697bb9f704becde9890 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:21.685534 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc1d9f-0f8c-47b5-b9b7-c8f261cf0866 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:21.686015 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc1d9f-0f8c-47b5-b9b7-c8f261cf0866 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:21.686155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc1d9f-0f8c-47b5-b9b7-c8f261cf0866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:21.709225 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-72dc1d9f-0f8c-47b5-b9b7-c8f261cf0866 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:21.711394 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-72dc1d9f-0f8c-47b5-b9b7-c8f261cf0866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:21.778631 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1404/2761] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:21.789531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a849f6e-4fc9-4def-8d77-0635ae6f3bab admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:21.790929 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7a849f6e-4fc9-4def-8d77-0635ae6f3bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:21.861697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a849f6e-4fc9-4def-8d77-0635ae6f3bab admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:21.868972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82efe8-136b-48c8-b2bd-fdddd1acd3c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:21.869817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82efe8-136b-48c8-b2bd-fdddd1acd3c6 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:21.870115 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82efe8-136b-48c8-b2bd-fdddd1acd3c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:21.870382 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82efe8-136b-48c8-b2bd-fdddd1acd3c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:21.880484 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82efe8-136b-48c8-b2bd-fdddd1acd3c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:21.881932 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82efe8-136b-48c8-b2bd-fdddd1acd3c6 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:21.892300 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1358/2762] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:38:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:38:22.190443 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc1d9f-0f8c-47b5-b9b7-c8f261cf0866 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:22.192157 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-72dc1d9f-0f8c-47b5-b9b7-c8f261cf0866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:22.205879 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc1d9f-0f8c-47b5-b9b7-c8f261cf0866 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:22.218038 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:22.228807 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:22.229105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:22.229398 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] PATH_INFO: `/v3/projects/dfa792a6828b476a962096e9ae012436` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:22.230796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfa792a6828b476a962096e9ae012436)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:22.233022 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:22.287318 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfa792a6828b476a962096e9ae012436', 'request_id': 'req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:22.293275 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfa792a6828b476a962096e9ae012436', 'request_id': 'req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:22.303547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfa792a6828b476a962096e9ae012436', 'request_id': 'req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:22.315536 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] The token cache is being invalidate because project dfa792a6828b476a962096e9ae012436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:38:22.315822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:22.316698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:22.319614 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1405/2763] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:38:22 2022] DELETE /identity/v3/projects/dfa792a6828b476a962096e9ae012436 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:22.323805 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82efe8-136b-48c8-b2bd-fdddd1acd3c6 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:22.334238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82efe8-136b-48c8-b2bd-fdddd1acd3c6 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:22.402319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502ca8a-ed16-43a0-a442-03cdf4852fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:22.403344 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502ca8a-ed16-43a0-a442-03cdf4852fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:22.403687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502ca8a-ed16-43a0-a442-03cdf4852fee admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:22.404033 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502ca8a-ed16-43a0-a442-03cdf4852fee admin admin] PATH_INFO: `/v3/users/0f87a01391d542cf9d02b8adb61bc730` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:22.405333 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502ca8a-ed16-43a0-a442-03cdf4852fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f87a01391d542cf9d02b8adb61bc730)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:22.407342 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502ca8a-ed16-43a0-a442-03cdf4852fee admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:22.459666 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-0502ca8a-ed16-43a0-a442-03cdf4852fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f87a01391d542cf9d02b8adb61bc730'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:22.469360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-0502ca8a-ed16-43a0-a442-03cdf4852fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f87a01391d542cf9d02b8adb61bc730'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:22.470012 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-0502ca8a-ed16-43a0-a442-03cdf4852fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f87a01391d542cf9d02b8adb61bc730'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:22.483027 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-0502ca8a-ed16-43a0-a442-03cdf4852fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f87a01391d542cf9d02b8adb61bc730'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:22.484007 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-0502ca8a-ed16-43a0-a442-03cdf4852fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f87a01391d542cf9d02b8adb61bc730'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:22.507738 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1359/2764] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:38:22 2022] DELETE /identity/v3/users/0f87a01391d542cf9d02b8adb61bc730 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:22.819734 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:22.825659 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:22.857051 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e1a6a-6194-47ae-ac00-a3b4be47a9b2 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:22.929495 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757fe5c-21a0-4583-a3e3-9d27a09337ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:22.930387 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757fe5c-21a0-4583-a3e3-9d27a09337ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:22.930656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757fe5c-21a0-4583-a3e3-9d27a09337ec admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:22.931120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757fe5c-21a0-4583-a3e3-9d27a09337ec admin admin] PATH_INFO: `/v3/projects/9b81db174a08400cab2c6a9637ceebb0` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:22.932677 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757fe5c-21a0-4583-a3e3-9d27a09337ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b81db174a08400cab2c6a9637ceebb0)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:22.934802 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757fe5c-21a0-4583-a3e3-9d27a09337ec admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:22.990828 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3757fe5c-21a0-4583-a3e3-9d27a09337ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b81db174a08400cab2c6a9637ceebb0', 'request_id': 'req-3757fe5c-21a0-4583-a3e3-9d27a09337ec'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:22.997683 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3757fe5c-21a0-4583-a3e3-9d27a09337ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b81db174a08400cab2c6a9637ceebb0', 'request_id': 'req-3757fe5c-21a0-4583-a3e3-9d27a09337ec'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:23.007126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3757fe5c-21a0-4583-a3e3-9d27a09337ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b81db174a08400cab2c6a9637ceebb0', 'request_id': 'req-3757fe5c-21a0-4583-a3e3-9d27a09337ec'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:23.019670 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3757fe5c-21a0-4583-a3e3-9d27a09337ec admin admin] The token cache is being invalidate because project 9b81db174a08400cab2c6a9637ceebb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:38:23.020071 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3757fe5c-21a0-4583-a3e3-9d27a09337ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:23.020754 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-3757fe5c-21a0-4583-a3e3-9d27a09337ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:23.022959 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1406/2765] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:38:22 2022] DELETE /identity/v3/projects/9b81db174a08400cab2c6a9637ceebb0 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:23.030638 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9714e7e-7dbb-4f0b-b964-ce0f1f52a376 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:23.031057 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9714e7e-7dbb-4f0b-b964-ce0f1f52a376 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:23.031448 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9714e7e-7dbb-4f0b-b964-ce0f1f52a376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:23.052814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-e9714e7e-7dbb-4f0b-b964-ce0f1f52a376 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:23.054208 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-e9714e7e-7dbb-4f0b-b964-ce0f1f52a376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:23.123029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1360/2766] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:23.131296 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae670f5-5af7-4636-bbbd-6d7029f4a34f None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:23.131755 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae670f5-5af7-4636-bbbd-6d7029f4a34f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:23.132041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae670f5-5af7-4636-bbbd-6d7029f4a34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:23.155699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-3ae670f5-5af7-4636-bbbd-6d7029f4a34f None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:23.156912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-3ae670f5-5af7-4636-bbbd-6d7029f4a34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:23.169021 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1407/2767] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:38:23.171230 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9714e7e-7dbb-4f0b-b964-ce0f1f52a376 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:23.186884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9714e7e-7dbb-4f0b-b964-ce0f1f52a376 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:23.192455 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b81e6-15db-4008-9575-9eb42b486606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:23.193129 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b81e6-15db-4008-9575-9eb42b486606 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:23.193416 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b81e6-15db-4008-9575-9eb42b486606 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:23.193730 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b81e6-15db-4008-9575-9eb42b486606 admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:23.194459 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b81e6-15db-4008-9575-9eb42b486606 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:38:23.194884 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b81e6-15db-4008-9575-9eb42b486606 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:23.196551 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b81e6-15db-4008-9575-9eb42b486606 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:23.218060 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1361/2768] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:38:23.219581 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae670f5-5af7-4636-bbbd-6d7029f4a34f None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:23.230224 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae670f5-5af7-4636-bbbd-6d7029f4a34f None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:23.235489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945915f-190e-4c2f-bd6d-a2f132aff346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:23.236121 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945915f-190e-4c2f-bd6d-a2f132aff346 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:23.236410 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945915f-190e-4c2f-bd6d-a2f132aff346 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:23.236687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945915f-190e-4c2f-bd6d-a2f132aff346 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:23.237438 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945915f-190e-4c2f-bd6d-a2f132aff346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:23.239426 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945915f-190e-4c2f-bd6d-a2f132aff346 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:23.267804 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1408/2769] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:38:23 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:38:23.272621 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b81e6-15db-4008-9575-9eb42b486606 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:23.283969 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b81e6-15db-4008-9575-9eb42b486606 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:23.355815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a32939-c96b-4bb4-a9b1-ecbbeee58220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:23.356661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a32939-c96b-4bb4-a9b1-ecbbeee58220 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:23.357059 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a32939-c96b-4bb4-a9b1-ecbbeee58220 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:23.357433 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a32939-c96b-4bb4-a9b1-ecbbeee58220 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:23.358218 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a32939-c96b-4bb4-a9b1-ecbbeee58220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:23.360291 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a32939-c96b-4bb4-a9b1-ecbbeee58220 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:23.385821 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1362/2770] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:38:23 2022] POST /identity/v3/users => generated 284 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:38:23.387094 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945915f-190e-4c2f-bd6d-a2f132aff346 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:23.398283 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945915f-190e-4c2f-bd6d-a2f132aff346 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:23.403426 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234cffd-b2b4-41aa-8911-390458877bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:23.404386 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234cffd-b2b4-41aa-8911-390458877bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:23.404992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234cffd-b2b4-41aa-8911-390458877bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:23.405480 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234cffd-b2b4-41aa-8911-390458877bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:23.406302 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234cffd-b2b4-41aa-8911-390458877bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:23.408054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234cffd-b2b4-41aa-8911-390458877bb9 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:23.422924 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1409/2771] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:38:23.426850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a32939-c96b-4bb4-a9b1-ecbbeee58220 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:23.437352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a32939-c96b-4bb4-a9b1-ecbbeee58220 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:23.442388 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec1952-93c7-42ba-b559-429a77c315f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:23.443082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec1952-93c7-42ba-b559-429a77c315f1 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:23.443417 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec1952-93c7-42ba-b559-429a77c315f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:23.443706 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec1952-93c7-42ba-b559-429a77c315f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:23.444442 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec1952-93c7-42ba-b559-429a77c315f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:23.446174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec1952-93c7-42ba-b559-429a77c315f1 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:23.459376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1363/2772] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:38:23.461751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234cffd-b2b4-41aa-8911-390458877bb9 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:23.472385 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234cffd-b2b4-41aa-8911-390458877bb9 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:23.479008 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e565b48-8bcb-43fb-994e-8becced9397c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:23.480015 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e565b48-8bcb-43fb-994e-8becced9397c admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:23.480367 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e565b48-8bcb-43fb-994e-8becced9397c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:23.480735 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e565b48-8bcb-43fb-994e-8becced9397c admin admin] PATH_INFO: `/v3/projects/2131f1d0e66c4f9a9c6d466d05ad3044/users/2d65bf385aad457c964f4ec20994e826/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:23.497553 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e565b48-8bcb-43fb-994e-8becced9397c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2131f1d0e66c4f9a9c6d466d05ad3044, user_id=2d65bf385aad457c964f4ec20994e826, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:23.500377 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e565b48-8bcb-43fb-994e-8becced9397c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:23.513392 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1410/2773] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:38:23 2022] PUT /identity/v3/projects/2131f1d0e66c4f9a9c6d466d05ad3044/users/2d65bf385aad457c964f4ec20994e826/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:38:23.525596 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e5844-c748-4c40-86f9-c82207c365e6 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:23.526082 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e5844-c748-4c40-86f9-c82207c365e6 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:23.526364 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e5844-c748-4c40-86f9-c82207c365e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:23.554501 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-781e5844-c748-4c40-86f9-c82207c365e6 None None] MFA Rules not processed for user `2d65bf385aad457c964f4ec20994e826`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:23.555747 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-781e5844-c748-4c40-86f9-c82207c365e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:23.624761 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1364/2774] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:23.630254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e565b48-8bcb-43fb-994e-8becced9397c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:23.717345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e565b48-8bcb-43fb-994e-8becced9397c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:23.722648 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d6b07-a329-434e-8003-2fbec6a46d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:23.723343 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d6b07-a329-434e-8003-2fbec6a46d2d admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:23.723642 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d6b07-a329-434e-8003-2fbec6a46d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:23.723905 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d6b07-a329-434e-8003-2fbec6a46d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:23.724938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d6b07-a329-434e-8003-2fbec6a46d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:23.727475 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d6b07-a329-434e-8003-2fbec6a46d2d admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:23.762232 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1411/2775] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:38:23 2022] POST /identity/v3/projects => generated 365 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:38:23.766435 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e5844-c748-4c40-86f9-c82207c365e6 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:23.778214 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e5844-c748-4c40-86f9-c82207c365e6 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:23.842692 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77023a98-b269-4357-9969-1cbb4735a86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:23.843496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77023a98-b269-4357-9969-1cbb4735a86a admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:23.843785 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77023a98-b269-4357-9969-1cbb4735a86a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:23.844064 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77023a98-b269-4357-9969-1cbb4735a86a admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:23.845101 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77023a98-b269-4357-9969-1cbb4735a86a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:23.847050 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77023a98-b269-4357-9969-1cbb4735a86a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:23.875309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1365/2776] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:38:23 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:38:23.876976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d6b07-a329-434e-8003-2fbec6a46d2d admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:23.888886 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d6b07-a329-434e-8003-2fbec6a46d2d admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:23.894632 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522986a5-4bd0-4a71-932d-e861a000cc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:23.895579 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522986a5-4bd0-4a71-932d-e861a000cc0d admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:23.895973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522986a5-4bd0-4a71-932d-e861a000cc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:23.896343 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522986a5-4bd0-4a71-932d-e861a000cc0d admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:23.900366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522986a5-4bd0-4a71-932d-e861a000cc0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:23.903150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522986a5-4bd0-4a71-932d-e861a000cc0d admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:23.916571 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1412/2777] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:38:23.923317 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77023a98-b269-4357-9969-1cbb4735a86a admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:23.935321 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77023a98-b269-4357-9969-1cbb4735a86a admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:23.944658 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd006f9f-54f4-4335-8376-2de2109ea99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:23.946648 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd006f9f-54f4-4335-8376-2de2109ea99e admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:23.947140 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd006f9f-54f4-4335-8376-2de2109ea99e admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:23.947863 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd006f9f-54f4-4335-8376-2de2109ea99e admin admin] PATH_INFO: `/v3/projects/cb99ccc04d464af5b6414273eabf094a/users/4e47f169441b43a088f920f1170d158c/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:23.974814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd006f9f-54f4-4335-8376-2de2109ea99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb99ccc04d464af5b6414273eabf094a, user_id=4e47f169441b43a088f920f1170d158c, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:23.977958 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd006f9f-54f4-4335-8376-2de2109ea99e admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:23.996054 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1366/2778] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:38:23 2022] PUT /identity/v3/projects/cb99ccc04d464af5b6414273eabf094a/users/4e47f169441b43a088f920f1170d158c/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:24.000280 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522986a5-4bd0-4a71-932d-e861a000cc0d admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:24.014146 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522986a5-4bd0-4a71-932d-e861a000cc0d admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:24.097489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7963c-ed04-49f5-9239-ad87847f6b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:24.098347 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7963c-ed04-49f5-9239-ad87847f6b56 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:24.098631 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7963c-ed04-49f5-9239-ad87847f6b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:24.098904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7963c-ed04-49f5-9239-ad87847f6b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:24.099682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7963c-ed04-49f5-9239-ad87847f6b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:24.101257 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7963c-ed04-49f5-9239-ad87847f6b56 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:24.115090 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1413/2779] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:38:24.119531 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd006f9f-54f4-4335-8376-2de2109ea99e admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:24.130752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd006f9f-54f4-4335-8376-2de2109ea99e admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:24.135466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61f39f-efee-4fb7-8f46-41d0d716f273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:24.136352 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61f39f-efee-4fb7-8f46-41d0d716f273 admin admin] REQUEST_METHOD: `PUT` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:24.136632 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61f39f-efee-4fb7-8f46-41d0d716f273 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:24.136910 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61f39f-efee-4fb7-8f46-41d0d716f273 admin admin] PATH_INFO: `/v3/projects/cb99ccc04d464af5b6414273eabf094a/users/4e47f169441b43a088f920f1170d158c/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:24.139071 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61f39f-efee-4fb7-8f46-41d0d716f273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb99ccc04d464af5b6414273eabf094a, user_id=4e47f169441b43a088f920f1170d158c, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:24.141752 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61f39f-efee-4fb7-8f46-41d0d716f273 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:24.153758 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1367/2780] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:38:24 2022] PUT /identity/v3/projects/cb99ccc04d464af5b6414273eabf094a/users/4e47f169441b43a088f920f1170d158c/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:24.165491 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa341e4-06ce-46a8-a533-b203d13689a9 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:24.166048 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa341e4-06ce-46a8-a533-b203d13689a9 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:24.166775 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa341e4-06ce-46a8-a533-b203d13689a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:24.200471 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-efa341e4-06ce-46a8-a533-b203d13689a9 None None] MFA Rules not processed for user `4e47f169441b43a088f920f1170d158c`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:24.201846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-efa341e4-06ce-46a8-a533-b203d13689a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:24.280225 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1414/2781] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:24 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:24.293212 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61f39f-efee-4fb7-8f46-41d0d716f273 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:24.294998 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-fd61f39f-efee-4fb7-8f46-41d0d716f273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:24.375520 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61f39f-efee-4fb7-8f46-41d0d716f273 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:24.381873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f1f21-f3cb-4acb-a846-260b1698daf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:24.382680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f1f21-f3cb-4acb-a846-260b1698daf1 service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:24.383065 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f1f21-f3cb-4acb-a846-260b1698daf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:24.383461 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f1f21-f3cb-4acb-a846-260b1698daf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:24.394936 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f1f21-f3cb-4acb-a846-260b1698daf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:24.396464 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f1f21-f3cb-4acb-a846-260b1698daf1 service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:24.406598 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1368/2782] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:38:24 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:38:24.510878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa341e4-06ce-46a8-a533-b203d13689a9 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:24.520927 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa341e4-06ce-46a8-a533-b203d13689a9 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:24.524728 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44eb7aa-2d44-4ad2-904c-3f9f1505bc40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:24.525404 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44eb7aa-2d44-4ad2-904c-3f9f1505bc40 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:24.525682 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44eb7aa-2d44-4ad2-904c-3f9f1505bc40 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:24.525955 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44eb7aa-2d44-4ad2-904c-3f9f1505bc40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:24.594615 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44eb7aa-2d44-4ad2-904c-3f9f1505bc40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:24.596062 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44eb7aa-2d44-4ad2-904c-3f9f1505bc40 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:24.607232 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1415/2783] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:38:24 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:38:29.709993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f1f21-f3cb-4acb-a846-260b1698daf1 service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:29.818263 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f1f21-f3cb-4acb-a846-260b1698daf1 service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:29.823304 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523cbab-f6e1-46b5-b9b6-126476e44a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:29.824277 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cbab-f6e1-46b5-b9b6-126476e44a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:29.824526 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cbab-f6e1-46b5-b9b6-126476e44a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:29.824925 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cbab-f6e1-46b5-b9b6-126476e44a7b admin admin] PATH_INFO: `/v3/users/2d65bf385aad457c964f4ec20994e826` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:29.826214 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523cbab-f6e1-46b5-b9b6-126476e44a7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d65bf385aad457c964f4ec20994e826)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:29.838745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523cbab-f6e1-46b5-b9b6-126476e44a7b admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:29.909415 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b523cbab-f6e1-46b5-b9b6-126476e44a7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d65bf385aad457c964f4ec20994e826'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:29.917101 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b523cbab-f6e1-46b5-b9b6-126476e44a7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d65bf385aad457c964f4ec20994e826'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:29.917832 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b523cbab-f6e1-46b5-b9b6-126476e44a7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d65bf385aad457c964f4ec20994e826'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:29.930311 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b523cbab-f6e1-46b5-b9b6-126476e44a7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d65bf385aad457c964f4ec20994e826'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:29.931032 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-b523cbab-f6e1-46b5-b9b6-126476e44a7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d65bf385aad457c964f4ec20994e826'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:29.955223 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1369/2784] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:38:29 2022] DELETE /identity/v3/users/2d65bf385aad457c964f4ec20994e826 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:29.958660 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0f36-087e-4af3-97b5-75d93d9339c3 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:29.959155 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0f36-087e-4af3-97b5-75d93d9339c3 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:29.959537 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0f36-087e-4af3-97b5-75d93d9339c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:29.988150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-1c4a0f36-087e-4af3-97b5-75d93d9339c3 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:29.989973 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-1c4a0f36-087e-4af3-97b5-75d93d9339c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:30.067549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1416/2785] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:30.077466 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523cbab-f6e1-46b5-b9b6-126476e44a7b admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:30.079272 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b523cbab-f6e1-46b5-b9b6-126476e44a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:30.146746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523cbab-f6e1-46b5-b9b6-126476e44a7b admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:30.150899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513c741-fb8e-42f1-8d4b-addd84e39450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:30.151850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513c741-fb8e-42f1-8d4b-addd84e39450 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:30.152141 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513c741-fb8e-42f1-8d4b-addd84e39450 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:30.152642 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513c741-fb8e-42f1-8d4b-addd84e39450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:30.164086 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513c741-fb8e-42f1-8d4b-addd84e39450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:30.165656 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513c741-fb8e-42f1-8d4b-addd84e39450 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:30.177422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1370/2786] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:38:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:38:30.520916 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a0f36-087e-4af3-97b5-75d93d9339c3 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:30.522478 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-1c4a0f36-087e-4af3-97b5-75d93d9339c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:30.538845 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a0f36-087e-4af3-97b5-75d93d9339c3 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:30.544529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:30.545423 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:30.545710 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:30.545993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] PATH_INFO: `/v3/projects/2131f1d0e66c4f9a9c6d466d05ad3044` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:30.547247 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] RBAC: Authorizing `identity:delete_project(project_id=2131f1d0e66c4f9a9c6d466d05ad3044)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:30.549342 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:30.598858 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513c741-fb8e-42f1-8d4b-addd84e39450 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:30.600512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-d513c741-fb8e-42f1-8d4b-addd84e39450 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:30.624056 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513c741-fb8e-42f1-8d4b-addd84e39450 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:30.633030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2131f1d0e66c4f9a9c6d466d05ad3044', 'request_id': 'req-38a6724d-169f-430b-b6e7-e49ec9f7d0db'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.638650 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2131f1d0e66c4f9a9c6d466d05ad3044', 'request_id': 'req-38a6724d-169f-430b-b6e7-e49ec9f7d0db'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.650495 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2131f1d0e66c4f9a9c6d466d05ad3044', 'request_id': 'req-38a6724d-169f-430b-b6e7-e49ec9f7d0db'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.666538 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] The token cache is being invalidate because project 2131f1d0e66c4f9a9c6d466d05ad3044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:38:30.666968 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.667774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.669621 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1417/2787] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:38:30 2022] DELETE /identity/v3/projects/2131f1d0e66c4f9a9c6d466d05ad3044 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:30.673863 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:30.675315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:30.701995 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81ff39-6aa2-4a20-b939-4573fdf79871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:30.703157 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81ff39-6aa2-4a20-b939-4573fdf79871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:30.703537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81ff39-6aa2-4a20-b939-4573fdf79871 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:30.703825 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81ff39-6aa2-4a20-b939-4573fdf79871 admin admin] PATH_INFO: `/v3/users/b2e8f8cb607649c0836a9c709791fc0e` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:30.706210 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81ff39-6aa2-4a20-b939-4573fdf79871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2e8f8cb607649c0836a9c709791fc0e)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:30.709091 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81ff39-6aa2-4a20-b939-4573fdf79871 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:30.756077 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6724d-169f-430b-b6e7-e49ec9f7d0db admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:30.761752 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cffc7-ed25-4ea0-b770-39df773cdcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:30.762335 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8b81ff39-6aa2-4a20-b939-4573fdf79871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2e8f8cb607649c0836a9c709791fc0e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.762803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cffc7-ed25-4ea0-b770-39df773cdcdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:30.762935 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cffc7-ed25-4ea0-b770-39df773cdcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:30.763169 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cffc7-ed25-4ea0-b770-39df773cdcdd admin admin] PATH_INFO: `/v3/users/4e47f169441b43a088f920f1170d158c` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:30.764413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cffc7-ed25-4ea0-b770-39df773cdcdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e47f169441b43a088f920f1170d158c)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:30.766315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cffc7-ed25-4ea0-b770-39df773cdcdd admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:30.770235 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8b81ff39-6aa2-4a20-b939-4573fdf79871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2e8f8cb607649c0836a9c709791fc0e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.771058 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8b81ff39-6aa2-4a20-b939-4573fdf79871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2e8f8cb607649c0836a9c709791fc0e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.781538 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8b81ff39-6aa2-4a20-b939-4573fdf79871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2e8f8cb607649c0836a9c709791fc0e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.782568 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-8b81ff39-6aa2-4a20-b939-4573fdf79871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2e8f8cb607649c0836a9c709791fc0e'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.802640 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1371/2788] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:38:30 2022] DELETE /identity/v3/users/b2e8f8cb607649c0836a9c709791fc0e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:38:30.834028 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-800cffc7-ed25-4ea0-b770-39df773cdcdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e47f169441b43a088f920f1170d158c'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.834862 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-800cffc7-ed25-4ea0-b770-39df773cdcdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e47f169441b43a088f920f1170d158c'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.845532 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-800cffc7-ed25-4ea0-b770-39df773cdcdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e47f169441b43a088f920f1170d158c'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.855777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-800cffc7-ed25-4ea0-b770-39df773cdcdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e47f169441b43a088f920f1170d158c'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.856917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-800cffc7-ed25-4ea0-b770-39df773cdcdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e47f169441b43a088f920f1170d158c'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:30.884576 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1418/2789] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:38:30 2022] DELETE /identity/v3/users/4e47f169441b43a088f920f1170d158c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:38:31.046231 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81ff39-6aa2-4a20-b939-4573fdf79871 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:31.051978 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-8b81ff39-6aa2-4a20-b939-4573fdf79871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:31.153093 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81ff39-6aa2-4a20-b939-4573fdf79871 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:31.219655 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:31.220675 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:31.221547 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:31.222105 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a admin admin] PATH_INFO: `/v3/projects/64dd788fefab4eeab253b0f0d341fd2d` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:31.223454 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=64dd788fefab4eeab253b0f0d341fd2d)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:31.225592 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:31.241974 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cffc7-ed25-4ea0-b770-39df773cdcdd admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:31.243883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-800cffc7-ed25-4ea0-b770-39df773cdcdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:31.260660 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cffc7-ed25-4ea0-b770-39df773cdcdd admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:31.267749 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e1117-2397-467b-a226-cf8e43d47f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:31.268846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e1117-2397-467b-a226-cf8e43d47f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:31.269126 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e1117-2397-467b-a226-cf8e43d47f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:31.269993 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e1117-2397-467b-a226-cf8e43d47f45 admin admin] PATH_INFO: `/v3/projects/cb99ccc04d464af5b6414273eabf094a` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:31.271191 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e1117-2397-467b-a226-cf8e43d47f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb99ccc04d464af5b6414273eabf094a)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:31.273093 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e1117-2397-467b-a226-cf8e43d47f45 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:31.308076 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64dd788fefab4eeab253b0f0d341fd2d', 'request_id': 'req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:31.317550 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64dd788fefab4eeab253b0f0d341fd2d', 'request_id': 'req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:31.334708 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64dd788fefab4eeab253b0f0d341fd2d', 'request_id': 'req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:31.346658 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a admin admin] The token cache is being invalidate because project 64dd788fefab4eeab253b0f0d341fd2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:38:31.346846 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:31.347727 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-bab9a990-f4a6-4bde-a64f-1b5630bc7a6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:31.349360 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1372/2790] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:38:31 2022] DELETE /identity/v3/projects/64dd788fefab4eeab253b0f0d341fd2d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:31.352686 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-145e1117-2397-467b-a226-cf8e43d47f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb99ccc04d464af5b6414273eabf094a', 'request_id': 'req-145e1117-2397-467b-a226-cf8e43d47f45'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:31.362393 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-145e1117-2397-467b-a226-cf8e43d47f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb99ccc04d464af5b6414273eabf094a', 'request_id': 'req-145e1117-2397-467b-a226-cf8e43d47f45'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:31.377471 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-145e1117-2397-467b-a226-cf8e43d47f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb99ccc04d464af5b6414273eabf094a', 'request_id': 'req-145e1117-2397-467b-a226-cf8e43d47f45'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:31.390117 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-145e1117-2397-467b-a226-cf8e43d47f45 admin admin] The token cache is being invalidate because project cb99ccc04d464af5b6414273eabf094a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:38:31.390409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-145e1117-2397-467b-a226-cf8e43d47f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:31.391314 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-145e1117-2397-467b-a226-cf8e43d47f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:31.393603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1419/2791] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:38:31 2022] DELETE /identity/v3/projects/cb99ccc04d464af5b6414273eabf094a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:31.398927 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac16a02-90f7-46a2-b35b-824ca845f972 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:31.399850 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac16a02-90f7-46a2-b35b-824ca845f972 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:31.400143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac16a02-90f7-46a2-b35b-824ca845f972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:31.422551 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-aac16a02-90f7-46a2-b35b-824ca845f972 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:31.423872 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-aac16a02-90f7-46a2-b35b-824ca845f972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:31.519777 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1373/2792] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:31.522623 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc426b-b1ec-490a-b0f7-029a9a12a8ee None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:31.523000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc426b-b1ec-490a-b0f7-029a9a12a8ee None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:31.523389 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc426b-b1ec-490a-b0f7-029a9a12a8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:31.555575 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-b9dc426b-b1ec-490a-b0f7-029a9a12a8ee None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:31.557060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-b9dc426b-b1ec-490a-b0f7-029a9a12a8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:31.567235 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1420/2793] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:31.572248 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac16a02-90f7-46a2-b35b-824ca845f972 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:31.585814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac16a02-90f7-46a2-b35b-824ca845f972 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:31.591026 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287fa532-8419-4162-8206-50ffc7450ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:31.592092 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fa532-8419-4162-8206-50ffc7450ade admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:31.592376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fa532-8419-4162-8206-50ffc7450ade admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:31.592711 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fa532-8419-4162-8206-50ffc7450ade admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:31.593551 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287fa532-8419-4162-8206-50ffc7450ade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:38:31.594114 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287fa532-8419-4162-8206-50ffc7450ade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:31.596998 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287fa532-8419-4162-8206-50ffc7450ade admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:31.609874 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1374/2794] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:38:31.614699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc426b-b1ec-490a-b0f7-029a9a12a8ee None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:31.626602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc426b-b1ec-490a-b0f7-029a9a12a8ee None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:31.631518 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09acf21-3fcc-4361-b898-5a5482b8ceb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:31.632684 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09acf21-3fcc-4361-b898-5a5482b8ceb9 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:31.632975 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09acf21-3fcc-4361-b898-5a5482b8ceb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:31.633255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09acf21-3fcc-4361-b898-5a5482b8ceb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:31.633943 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09acf21-3fcc-4361-b898-5a5482b8ceb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:31.635859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09acf21-3fcc-4361-b898-5a5482b8ceb9 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:31.666496 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1421/2795] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:38:31 2022] POST /identity/v3/projects => generated 359 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:38:31.670652 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287fa532-8419-4162-8206-50ffc7450ade admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:31.686051 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287fa532-8419-4162-8206-50ffc7450ade admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:31.753663 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8aff82-7db2-4d9c-ac16-d2403867e2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:31.754475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8aff82-7db2-4d9c-ac16-d2403867e2bc admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:31.754764 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8aff82-7db2-4d9c-ac16-d2403867e2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:31.755034 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8aff82-7db2-4d9c-ac16-d2403867e2bc admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:31.755956 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8aff82-7db2-4d9c-ac16-d2403867e2bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:31.757865 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8aff82-7db2-4d9c-ac16-d2403867e2bc admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:31.785468 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1375/2796] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:38:31 2022] POST /identity/v3/users => generated 280 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:38:31.789656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09acf21-3fcc-4361-b898-5a5482b8ceb9 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:31.800557 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09acf21-3fcc-4361-b898-5a5482b8ceb9 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:31.807489 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4030d-7f5d-49d8-9ba3-9be47c443171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:31.808375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4030d-7f5d-49d8-9ba3-9be47c443171 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:31.808832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4030d-7f5d-49d8-9ba3-9be47c443171 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:31.809272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4030d-7f5d-49d8-9ba3-9be47c443171 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:31.810175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4030d-7f5d-49d8-9ba3-9be47c443171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:31.811965 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4030d-7f5d-49d8-9ba3-9be47c443171 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:31.827218 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1422/2797] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:38:31.829029 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8aff82-7db2-4d9c-ac16-d2403867e2bc admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:31.843554 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8aff82-7db2-4d9c-ac16-d2403867e2bc admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:31.848532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23e9c2-573c-4e6d-b67c-d05bca317603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:31.849408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23e9c2-573c-4e6d-b67c-d05bca317603 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:31.849786 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23e9c2-573c-4e6d-b67c-d05bca317603 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:31.850176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23e9c2-573c-4e6d-b67c-d05bca317603 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:31.851014 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23e9c2-573c-4e6d-b67c-d05bca317603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:31.852714 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23e9c2-573c-4e6d-b67c-d05bca317603 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:31.863556 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1376/2798] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:38:31.867784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4030d-7f5d-49d8-9ba3-9be47c443171 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:31.879027 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4030d-7f5d-49d8-9ba3-9be47c443171 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:31.883819 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200342d-32b5-4dc7-80a0-bfb781955f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:31.884715 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200342d-32b5-4dc7-80a0-bfb781955f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:31.884997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200342d-32b5-4dc7-80a0-bfb781955f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:31.885279 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200342d-32b5-4dc7-80a0-bfb781955f0e admin admin] PATH_INFO: `/v3/projects/4d5535e40efe4387a10f3540cd344c3e/users/baa10b5e36bd4d29b7925990a6dbd4bb/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:31.903793 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200342d-32b5-4dc7-80a0-bfb781955f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5535e40efe4387a10f3540cd344c3e, user_id=baa10b5e36bd4d29b7925990a6dbd4bb, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:31.906658 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200342d-32b5-4dc7-80a0-bfb781955f0e admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:31.922838 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1423/2799] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:38:31 2022] PUT /identity/v3/projects/4d5535e40efe4387a10f3540cd344c3e/users/baa10b5e36bd4d29b7925990a6dbd4bb/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:31.933572 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb332f-3703-449c-a7ff-7394daf4a96f None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:31.933847 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb332f-3703-449c-a7ff-7394daf4a96f None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:31.934125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb332f-3703-449c-a7ff-7394daf4a96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:31.961834 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-defb332f-3703-449c-a7ff-7394daf4a96f None None] MFA Rules not processed for user `baa10b5e36bd4d29b7925990a6dbd4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:31.963147 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-defb332f-3703-449c-a7ff-7394daf4a96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:32.035481 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1377/2800] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:31 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:32.067907 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200342d-32b5-4dc7-80a0-bfb781955f0e admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:32.069614 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-4200342d-32b5-4dc7-80a0-bfb781955f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:32.130732 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200342d-32b5-4dc7-80a0-bfb781955f0e admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:32.134407 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35ecf7-d839-4041-bf95-a04e369ce685 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:32.135056 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35ecf7-d839-4041-bf95-a04e369ce685 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:32.135361 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35ecf7-d839-4041-bf95-a04e369ce685 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:32.135635 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35ecf7-d839-4041-bf95-a04e369ce685 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:32.144341 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35ecf7-d839-4041-bf95-a04e369ce685 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:32.145739 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35ecf7-d839-4041-bf95-a04e369ce685 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:32.154769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1424/2801] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:38:32 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:38:32.168912 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb332f-3703-449c-a7ff-7394daf4a96f None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:32.234342 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb332f-3703-449c-a7ff-7394daf4a96f None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:32.239818 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8a05b-1931-48ee-894a-39d954f2d34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:32.240612 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8a05b-1931-48ee-894a-39d954f2d34c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:32.240892 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8a05b-1931-48ee-894a-39d954f2d34c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:32.241164 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8a05b-1931-48ee-894a-39d954f2d34c admin admin] PATH_INFO: `/v3/users/baa10b5e36bd4d29b7925990a6dbd4bb` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:32.242359 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8a05b-1931-48ee-894a-39d954f2d34c admin admin] RBAC: Authorizing `identity:delete_user(user_id=baa10b5e36bd4d29b7925990a6dbd4bb)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:32.244600 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8a05b-1931-48ee-894a-39d954f2d34c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:32.288281 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f7a8a05b-1931-48ee-894a-39d954f2d34c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baa10b5e36bd4d29b7925990a6dbd4bb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:32.295000 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f7a8a05b-1931-48ee-894a-39d954f2d34c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baa10b5e36bd4d29b7925990a6dbd4bb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:32.295887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f7a8a05b-1931-48ee-894a-39d954f2d34c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baa10b5e36bd4d29b7925990a6dbd4bb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:32.303627 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f7a8a05b-1931-48ee-894a-39d954f2d34c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baa10b5e36bd4d29b7925990a6dbd4bb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:32.304226 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f7a8a05b-1931-48ee-894a-39d954f2d34c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baa10b5e36bd4d29b7925990a6dbd4bb'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:32.322132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1378/2802] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:38:32 2022] DELETE /identity/v3/users/baa10b5e36bd4d29b7925990a6dbd4bb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:32.326606 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e1d08-8c1d-4607-bcd6-277071fbb4b6 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:32.327098 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e1d08-8c1d-4607-bcd6-277071fbb4b6 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:32.327231 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e1d08-8c1d-4607-bcd6-277071fbb4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:32.343763 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-0a8e1d08-8c1d-4607-bcd6-277071fbb4b6 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:32.344947 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-0a8e1d08-8c1d-4607-bcd6-277071fbb4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:32.401723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1425/2803] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:38:32.411271 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8a05b-1931-48ee-894a-39d954f2d34c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:32.412745 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f7a8a05b-1931-48ee-894a-39d954f2d34c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:32.467063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8a05b-1931-48ee-894a-39d954f2d34c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:32.470642 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747bce9-d831-4c4e-99ef-50ceb32e9ba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:32.471380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747bce9-d831-4c4e-99ef-50ceb32e9ba8 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:32.471665 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747bce9-d831-4c4e-99ef-50ceb32e9ba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:32.471938 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747bce9-d831-4c4e-99ef-50ceb32e9ba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:32.480588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747bce9-d831-4c4e-99ef-50ceb32e9ba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:32.481989 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747bce9-d831-4c4e-99ef-50ceb32e9ba8 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:32.492696 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1379/2804] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:38:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:38:32.746390 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e1d08-8c1d-4607-bcd6-277071fbb4b6 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:32.747811 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-0a8e1d08-8c1d-4607-bcd6-277071fbb4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:32.761209 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e1d08-8c1d-4607-bcd6-277071fbb4b6 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:32.766113 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080c333-3a0f-410d-a95d-b4e449a81b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:32.767066 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080c333-3a0f-410d-a95d-b4e449a81b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:32.767596 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080c333-3a0f-410d-a95d-b4e449a81b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:32.767951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080c333-3a0f-410d-a95d-b4e449a81b5a admin admin] PATH_INFO: `/v3/projects/4d5535e40efe4387a10f3540cd344c3e` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:32.769081 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080c333-3a0f-410d-a95d-b4e449a81b5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d5535e40efe4387a10f3540cd344c3e)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:32.770987 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080c333-3a0f-410d-a95d-b4e449a81b5a admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:32.817105 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c080c333-3a0f-410d-a95d-b4e449a81b5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d5535e40efe4387a10f3540cd344c3e', 'request_id': 'req-c080c333-3a0f-410d-a95d-b4e449a81b5a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:32.822312 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c080c333-3a0f-410d-a95d-b4e449a81b5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d5535e40efe4387a10f3540cd344c3e', 'request_id': 'req-c080c333-3a0f-410d-a95d-b4e449a81b5a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:32.829005 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c080c333-3a0f-410d-a95d-b4e449a81b5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d5535e40efe4387a10f3540cd344c3e', 'request_id': 'req-c080c333-3a0f-410d-a95d-b4e449a81b5a'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:32.839574 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c080c333-3a0f-410d-a95d-b4e449a81b5a admin admin] The token cache is being invalidate because project 4d5535e40efe4387a10f3540cd344c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:38:32.839857 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c080c333-3a0f-410d-a95d-b4e449a81b5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:32.840477 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-c080c333-3a0f-410d-a95d-b4e449a81b5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:32.841989 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1426/2805] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:38:32 2022] DELETE /identity/v3/projects/4d5535e40efe4387a10f3540cd344c3e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:32.847437 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56bee8-a232-44c4-9137-9f96d016fffc None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:32.847941 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56bee8-a232-44c4-9137-9f96d016fffc None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:32.848052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56bee8-a232-44c4-9137-9f96d016fffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:32.865755 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-aa56bee8-a232-44c4-9137-9f96d016fffc None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:32.866934 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-aa56bee8-a232-44c4-9137-9f96d016fffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:32.923429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1380/2806] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:32.932851 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26831d-b20d-4c41-94a3-1a75ce31c104 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:32.933145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26831d-b20d-4c41-94a3-1a75ce31c104 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:32.933430 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26831d-b20d-4c41-94a3-1a75ce31c104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:32.951149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-7e26831d-b20d-4c41-94a3-1a75ce31c104 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:32.952359 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-7e26831d-b20d-4c41-94a3-1a75ce31c104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:32.962029 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1427/2807] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:38:32.966680 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56bee8-a232-44c4-9137-9f96d016fffc None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:32.976894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56bee8-a232-44c4-9137-9f96d016fffc None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:32.981385 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3455b-0277-408e-9b74-2c57dff28ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:32.982042 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3455b-0277-408e-9b74-2c57dff28ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:32.982319 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3455b-0277-408e-9b74-2c57dff28ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:32.982588 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3455b-0277-408e-9b74-2c57dff28ad8 admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:32.983331 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3455b-0277-408e-9b74-2c57dff28ad8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:38:32.983635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3455b-0277-408e-9b74-2c57dff28ad8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:32.985445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3455b-0277-408e-9b74-2c57dff28ad8 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:32.999407 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1381/2808] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:38:33.001569 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26831d-b20d-4c41-94a3-1a75ce31c104 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:33.010789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26831d-b20d-4c41-94a3-1a75ce31c104 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:33.015497 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf366d-257c-4ae7-aee2-b6efabff3dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:33.016154 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf366d-257c-4ae7-aee2-b6efabff3dca admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:33.016439 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf366d-257c-4ae7-aee2-b6efabff3dca admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:33.016834 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf366d-257c-4ae7-aee2-b6efabff3dca admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:33.017441 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf366d-257c-4ae7-aee2-b6efabff3dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:33.019884 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf366d-257c-4ae7-aee2-b6efabff3dca admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:33.028733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1aa49-c41e-4aac-9410-03dc1b03ce52 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:33.029028 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1aa49-c41e-4aac-9410-03dc1b03ce52 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:33.029305 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1aa49-c41e-4aac-9410-03dc1b03ce52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:33.048873 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1428/2809] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:38:33 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:38:33.049117 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-c6c1aa49-c41e-4aac-9410-03dc1b03ce52 None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:33.049717 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c6c1aa49-c41e-4aac-9410-03dc1b03ce52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:33.050273 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf366d-257c-4ae7-aee2-b6efabff3dca admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:33.105438 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1382/2810] 10.222.0.25 () {58 vars in 967 bytes} [Mon May 9 18:38:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:38:33.109634 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf366d-257c-4ae7-aee2-b6efabff3dca admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:33.114293 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefc54b-5f29-4566-bb64-039d61bac23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:33.114951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefc54b-5f29-4566-bb64-039d61bac23e admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:33.115262 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefc54b-5f29-4566-bb64-039d61bac23e admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:33.115549 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefc54b-5f29-4566-bb64-039d61bac23e admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:33.116215 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebefc54b-5f29-4566-bb64-039d61bac23e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:33.117828 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1aa49-c41e-4aac-9410-03dc1b03ce52 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:33.118450 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebefc54b-5f29-4566-bb64-039d61bac23e admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:33.120049 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-c6c1aa49-c41e-4aac-9410-03dc1b03ce52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:33.138413 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1429/2811] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:38:33 2022] POST /identity/v3/users => generated 288 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:38:33.142434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefc54b-5f29-4566-bb64-039d61bac23e admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:33.152488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefc54b-5f29-4566-bb64-039d61bac23e admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:33.156997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c2aa8-6149-4a7f-84ab-3824b889e2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:33.157630 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c2aa8-6149-4a7f-84ab-3824b889e2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:33.157904 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c2aa8-6149-4a7f-84ab-3824b889e2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:33.158175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c2aa8-6149-4a7f-84ab-3824b889e2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:33.159488 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c2aa8-6149-4a7f-84ab-3824b889e2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:33.161046 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c2aa8-6149-4a7f-84ab-3824b889e2b8 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:33.170638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1430/2812] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:38:33.174529 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c2aa8-6149-4a7f-84ab-3824b889e2b8 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:33.183866 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c2aa8-6149-4a7f-84ab-3824b889e2b8 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:33.189211 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95eb38-bb7f-41d4-8017-ee10298523bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:33.189859 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95eb38-bb7f-41d4-8017-ee10298523bc admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:33.190022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1aa49-c41e-4aac-9410-03dc1b03ce52 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:33.190140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95eb38-bb7f-41d4-8017-ee10298523bc admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:33.190404 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95eb38-bb7f-41d4-8017-ee10298523bc admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:33.191130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95eb38-bb7f-41d4-8017-ee10298523bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:33.192691 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95eb38-bb7f-41d4-8017-ee10298523bc admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:33.194996 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ed7f8-9776-4156-99e1-5d0a10c2b494 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:33.195668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ed7f8-9776-4156-99e1-5d0a10c2b494 service swift] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:33.195947 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ed7f8-9776-4156-99e1-5d0a10c2b494 service swift] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:33.196219 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ed7f8-9776-4156-99e1-5d0a10c2b494 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:33.206288 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1431/2813] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:38:33.206458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ed7f8-9776-4156-99e1-5d0a10c2b494 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:33.206458 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ed7f8-9776-4156-99e1-5d0a10c2b494 service swift] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:33.207250 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95eb38-bb7f-41d4-8017-ee10298523bc admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:33.214505 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1383/2814] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:38:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 18:38:33.216603 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95eb38-bb7f-41d4-8017-ee10298523bc admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:33.221910 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f045bf-b26d-447d-98f8-8e1478dc849b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:33.222784 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f045bf-b26d-447d-98f8-8e1478dc849b admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:33.223064 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f045bf-b26d-447d-98f8-8e1478dc849b admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:33.223366 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f045bf-b26d-447d-98f8-8e1478dc849b admin admin] PATH_INFO: `/v3/projects/7d186503e14a4674bebd028d10b3726b/users/bf8348f7d99b4fdb892d25571a6d1202/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:33.238255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f045bf-b26d-447d-98f8-8e1478dc849b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d186503e14a4674bebd028d10b3726b, user_id=bf8348f7d99b4fdb892d25571a6d1202, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:33.241043 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f045bf-b26d-447d-98f8-8e1478dc849b admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:33.254327 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1432/2815] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:38:33 2022] PUT /identity/v3/projects/7d186503e14a4674bebd028d10b3726b/users/bf8348f7d99b4fdb892d25571a6d1202/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:38:33.320063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adcf6d-0ccc-4504-b068-a97e09904689 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:33.320351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adcf6d-0ccc-4504-b068-a97e09904689 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:33.320701 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adcf6d-0ccc-4504-b068-a97e09904689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:33.348707 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-41adcf6d-0ccc-4504-b068-a97e09904689 None None] MFA Rules not processed for user `bf8348f7d99b4fdb892d25571a6d1202`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:33.349886 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-41adcf6d-0ccc-4504-b068-a97e09904689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:33.416567 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1384/2816] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:33.428499 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f045bf-b26d-447d-98f8-8e1478dc849b admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:33.430357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-63f045bf-b26d-447d-98f8-8e1478dc849b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:33.489767 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f045bf-b26d-447d-98f8-8e1478dc849b admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:33.496082 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3d102-a81c-4f55-ab31-24ff2513ca1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:33.496761 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3d102-a81c-4f55-ab31-24ff2513ca1c service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:33.497153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3d102-a81c-4f55-ab31-24ff2513ca1c service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:33.497503 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3d102-a81c-4f55-ab31-24ff2513ca1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:33.511626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3d102-a81c-4f55-ab31-24ff2513ca1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:33.514406 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3d102-a81c-4f55-ab31-24ff2513ca1c service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:33.528010 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1433/2817] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:38:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:38:35.133867 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adcf6d-0ccc-4504-b068-a97e09904689 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:35.135507 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-41adcf6d-0ccc-4504-b068-a97e09904689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:35.199086 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adcf6d-0ccc-4504-b068-a97e09904689 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:35.203999 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b0716-4c18-4c31-9a50-22cc8051e2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:35.204873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b0716-4c18-4c31-9a50-22cc8051e2ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:35.205155 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b0716-4c18-4c31-9a50-22cc8051e2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:35.205972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b0716-4c18-4c31-9a50-22cc8051e2ce admin admin] PATH_INFO: `/v3/users/cdb5118d48d0412f8536c47b9dffb88c` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:35.207154 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b0716-4c18-4c31-9a50-22cc8051e2ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdb5118d48d0412f8536c47b9dffb88c)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:35.209106 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b0716-4c18-4c31-9a50-22cc8051e2ce admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:35.252393 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-201b0716-4c18-4c31-9a50-22cc8051e2ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdb5118d48d0412f8536c47b9dffb88c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:35.259038 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-201b0716-4c18-4c31-9a50-22cc8051e2ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdb5118d48d0412f8536c47b9dffb88c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:35.259733 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-201b0716-4c18-4c31-9a50-22cc8051e2ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdb5118d48d0412f8536c47b9dffb88c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:35.266807 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-201b0716-4c18-4c31-9a50-22cc8051e2ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdb5118d48d0412f8536c47b9dffb88c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:35.267459 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-201b0716-4c18-4c31-9a50-22cc8051e2ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdb5118d48d0412f8536c47b9dffb88c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:35.285126 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1385/2818] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:38:35 2022] DELETE /identity/v3/users/cdb5118d48d0412f8536c47b9dffb88c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:35.447853 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3d102-a81c-4f55-ab31-24ff2513ca1c service cinder] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:35.458938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3d102-a81c-4f55-ab31-24ff2513ca1c service cinder] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:35.516441 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95865968-c82d-49b1-b3bb-b81257bde402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:35.517308 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95865968-c82d-49b1-b3bb-b81257bde402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:35.517617 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95865968-c82d-49b1-b3bb-b81257bde402 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:35.518440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95865968-c82d-49b1-b3bb-b81257bde402 admin admin] PATH_INFO: `/v3/projects/05197c0440624fce9090b97e6826d516` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:35.519715 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95865968-c82d-49b1-b3bb-b81257bde402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05197c0440624fce9090b97e6826d516)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:35.521593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95865968-c82d-49b1-b3bb-b81257bde402 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:35.566826 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-95865968-c82d-49b1-b3bb-b81257bde402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05197c0440624fce9090b97e6826d516', 'request_id': 'req-95865968-c82d-49b1-b3bb-b81257bde402'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:35.572100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-95865968-c82d-49b1-b3bb-b81257bde402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05197c0440624fce9090b97e6826d516', 'request_id': 'req-95865968-c82d-49b1-b3bb-b81257bde402'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:35.579870 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-95865968-c82d-49b1-b3bb-b81257bde402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05197c0440624fce9090b97e6826d516', 'request_id': 'req-95865968-c82d-49b1-b3bb-b81257bde402'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:35.590809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-95865968-c82d-49b1-b3bb-b81257bde402 admin admin] The token cache is being invalidate because project 05197c0440624fce9090b97e6826d516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:38:35.591096 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-95865968-c82d-49b1-b3bb-b81257bde402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:35.591744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-95865968-c82d-49b1-b3bb-b81257bde402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:35.593272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1434/2819] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:38:35 2022] DELETE /identity/v3/projects/05197c0440624fce9090b97e6826d516 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:35.598835 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ddbd6-614c-48e5-8008-6773a13d273a None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:35.599422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ddbd6-614c-48e5-8008-6773a13d273a None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:35.599422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ddbd6-614c-48e5-8008-6773a13d273a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:35.616984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-b10ddbd6-614c-48e5-8008-6773a13d273a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:35.618420 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-b10ddbd6-614c-48e5-8008-6773a13d273a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:35.676862 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1386/2820] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:35.684274 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacd645-fbd4-4237-803b-1746439b393a None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:35.684567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacd645-fbd4-4237-803b-1746439b393a None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:35.684848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feacd645-fbd4-4237-803b-1746439b393a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:35.702912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-feacd645-fbd4-4237-803b-1746439b393a None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:35.704124 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-feacd645-fbd4-4237-803b-1746439b393a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:35.712454 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1435/2821] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:35.716915 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ddbd6-614c-48e5-8008-6773a13d273a None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:35.726309 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ddbd6-614c-48e5-8008-6773a13d273a None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:35.730937 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f43b7a-992a-424f-af39-01bfdd5645db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:35.731628 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f43b7a-992a-424f-af39-01bfdd5645db admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:35.731912 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f43b7a-992a-424f-af39-01bfdd5645db admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:35.732183 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f43b7a-992a-424f-af39-01bfdd5645db admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:35.732899 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f43b7a-992a-424f-af39-01bfdd5645db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:38:35.733203 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f43b7a-992a-424f-af39-01bfdd5645db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:35.734766 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f43b7a-992a-424f-af39-01bfdd5645db admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:35.746347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1387/2822] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 18:38:35.750567 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feacd645-fbd4-4237-803b-1746439b393a None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:35.760181 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feacd645-fbd4-4237-803b-1746439b393a None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:35.764777 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17498875-f00a-46fb-b42d-079efeca2bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:35.765419 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17498875-f00a-46fb-b42d-079efeca2bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:35.765702 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17498875-f00a-46fb-b42d-079efeca2bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:35.765975 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17498875-f00a-46fb-b42d-079efeca2bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:35.766657 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17498875-f00a-46fb-b42d-079efeca2bf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:35.769150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17498875-f00a-46fb-b42d-079efeca2bf3 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:35.793186 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1436/2823] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:38:35 2022] POST /identity/v3/projects => generated 373 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:38:35.797498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f43b7a-992a-424f-af39-01bfdd5645db admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:35.807224 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f43b7a-992a-424f-af39-01bfdd5645db admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:35.861532 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe2a8f-0e8c-428e-a4b8-72cc10bd98ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:35.862201 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe2a8f-0e8c-428e-a4b8-72cc10bd98ca admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:35.862483 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe2a8f-0e8c-428e-a4b8-72cc10bd98ca admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:35.862757 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe2a8f-0e8c-428e-a4b8-72cc10bd98ca admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:35.863475 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe2a8f-0e8c-428e-a4b8-72cc10bd98ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:35.865386 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe2a8f-0e8c-428e-a4b8-72cc10bd98ca admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:35.884622 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1388/2824] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:38:35 2022] POST /identity/v3/users => generated 287 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:38:35.888690 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17498875-f00a-46fb-b42d-079efeca2bf3 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:35.899575 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17498875-f00a-46fb-b42d-079efeca2bf3 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:35.904456 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b72e0-cc6f-4e1e-89ee-719e12925d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:35.905108 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b72e0-cc6f-4e1e-89ee-719e12925d7c admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:35.905390 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b72e0-cc6f-4e1e-89ee-719e12925d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:35.905662 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b72e0-cc6f-4e1e-89ee-719e12925d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:35.906378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b72e0-cc6f-4e1e-89ee-719e12925d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:35.907997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b72e0-cc6f-4e1e-89ee-719e12925d7c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:35.918026 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1437/2825] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:38:35.921814 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe2a8f-0e8c-428e-a4b8-72cc10bd98ca admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:35.931391 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe2a8f-0e8c-428e-a4b8-72cc10bd98ca admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:35.936150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61085818-46ed-4c84-b7aa-45f791e5ed29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:35.936797 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61085818-46ed-4c84-b7aa-45f791e5ed29 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:35.937077 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61085818-46ed-4c84-b7aa-45f791e5ed29 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:35.937351 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61085818-46ed-4c84-b7aa-45f791e5ed29 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:35.938061 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61085818-46ed-4c84-b7aa-45f791e5ed29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:35.940143 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61085818-46ed-4c84-b7aa-45f791e5ed29 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:35.949574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1389/2826] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 18:38:35.953244 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b72e0-cc6f-4e1e-89ee-719e12925d7c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:35.963518 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b72e0-cc6f-4e1e-89ee-719e12925d7c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:35.968136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db690885-0706-4870-94bc-615ab84bc802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:35.969020 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db690885-0706-4870-94bc-615ab84bc802 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:35.969296 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db690885-0706-4870-94bc-615ab84bc802 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:35.969568 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db690885-0706-4870-94bc-615ab84bc802 admin admin] PATH_INFO: `/v3/projects/de52a59737cf465489cd0b47a464a1d3/users/8a580323dec54be78300f0e7b7ca9410/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:35.983540 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db690885-0706-4870-94bc-615ab84bc802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de52a59737cf465489cd0b47a464a1d3, user_id=8a580323dec54be78300f0e7b7ca9410, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:35.986145 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db690885-0706-4870-94bc-615ab84bc802 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:35.999390 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1438/2827] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:38:35 2022] PUT /identity/v3/projects/de52a59737cf465489cd0b47a464a1d3/users/8a580323dec54be78300f0e7b7ca9410/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:36.010357 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f1330-e3c3-4188-942b-5aaa502929f7 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:36.010651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f1330-e3c3-4188-942b-5aaa502929f7 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:36.010930 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f1330-e3c3-4188-942b-5aaa502929f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:36.036651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-bc5f1330-e3c3-4188-942b-5aaa502929f7 None None] MFA Rules not processed for user `8a580323dec54be78300f0e7b7ca9410`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:36.037837 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-bc5f1330-e3c3-4188-942b-5aaa502929f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:36.099201 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1390/2828] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:36.110163 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db690885-0706-4870-94bc-615ab84bc802 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:36.112357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-db690885-0706-4870-94bc-615ab84bc802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:36.166324 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db690885-0706-4870-94bc-615ab84bc802 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:36.169928 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b3cf3-f2f5-41e9-8746-6260aa2556a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:36.170558 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b3cf3-f2f5-41e9-8746-6260aa2556a9 service cinder] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:36.170832 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b3cf3-f2f5-41e9-8746-6260aa2556a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:36.171103 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b3cf3-f2f5-41e9-8746-6260aa2556a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:36.179559 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b3cf3-f2f5-41e9-8746-6260aa2556a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:36.180950 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b3cf3-f2f5-41e9-8746-6260aa2556a9 service cinder] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:36.189378 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1439/2829] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:38:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:38:50.025603 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f1330-e3c3-4188-942b-5aaa502929f7 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:50.085577 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f1330-e3c3-4188-942b-5aaa502929f7 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:50.090085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b46ab-1aa5-4eb3-a851-77d41134ab8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:50.090955 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b46ab-1aa5-4eb3-a851-77d41134ab8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:50.091271 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b46ab-1aa5-4eb3-a851-77d41134ab8c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:50.091548 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b46ab-1aa5-4eb3-a851-77d41134ab8c admin admin] PATH_INFO: `/v3/users/8a580323dec54be78300f0e7b7ca9410` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:50.092713 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b46ab-1aa5-4eb3-a851-77d41134ab8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a580323dec54be78300f0e7b7ca9410)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:50.094611 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b46ab-1aa5-4eb3-a851-77d41134ab8c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:50.140308 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f27b46ab-1aa5-4eb3-a851-77d41134ab8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a580323dec54be78300f0e7b7ca9410'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:50.146857 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f27b46ab-1aa5-4eb3-a851-77d41134ab8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a580323dec54be78300f0e7b7ca9410'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:50.147537 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f27b46ab-1aa5-4eb3-a851-77d41134ab8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a580323dec54be78300f0e7b7ca9410'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:50.154852 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f27b46ab-1aa5-4eb3-a851-77d41134ab8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a580323dec54be78300f0e7b7ca9410'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:50.155504 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-f27b46ab-1aa5-4eb3-a851-77d41134ab8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a580323dec54be78300f0e7b7ca9410'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:50.173342 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1391/2830] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:38:50 2022] DELETE /identity/v3/users/8a580323dec54be78300f0e7b7ca9410 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:50.179551 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6525df-3f96-4ea7-b12e-d44ea68dbcbb None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:50.180030 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6525df-3f96-4ea7-b12e-d44ea68dbcbb None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:50.180130 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6525df-3f96-4ea7-b12e-d44ea68dbcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:50.199471 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-bc6525df-3f96-4ea7-b12e-d44ea68dbcbb None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:50.200756 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-bc6525df-3f96-4ea7-b12e-d44ea68dbcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:50.258206 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1440/2831] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:50.267638 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b46ab-1aa5-4eb3-a851-77d41134ab8c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:50.269488 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-f27b46ab-1aa5-4eb3-a851-77d41134ab8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:50.323574 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b46ab-1aa5-4eb3-a851-77d41134ab8c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:50.327104 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6727842-7c8b-4c14-9a21-6d559e5a051d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:50.327770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6727842-7c8b-4c14-9a21-6d559e5a051d service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:50.328052 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6727842-7c8b-4c14-9a21-6d559e5a051d service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:50.328327 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6727842-7c8b-4c14-9a21-6d559e5a051d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:50.337936 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6727842-7c8b-4c14-9a21-6d559e5a051d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:50.339399 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6727842-7c8b-4c14-9a21-6d559e5a051d service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:50.349215 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1392/2832] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:38:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:38:50.602288 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6525df-3f96-4ea7-b12e-d44ea68dbcbb None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:50.603836 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-bc6525df-3f96-4ea7-b12e-d44ea68dbcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:50.615789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6525df-3f96-4ea7-b12e-d44ea68dbcbb None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:50.621065 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c418a-e0d8-4c47-968c-f1a447688233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:50.622007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c418a-e0d8-4c47-968c-f1a447688233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:50.622361 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c418a-e0d8-4c47-968c-f1a447688233 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:50.622638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c418a-e0d8-4c47-968c-f1a447688233 admin admin] PATH_INFO: `/v3/projects/de52a59737cf465489cd0b47a464a1d3` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:50.623850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c418a-e0d8-4c47-968c-f1a447688233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de52a59737cf465489cd0b47a464a1d3)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:50.625725 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c418a-e0d8-4c47-968c-f1a447688233 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:50.672083 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ec9c418a-e0d8-4c47-968c-f1a447688233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de52a59737cf465489cd0b47a464a1d3', 'request_id': 'req-ec9c418a-e0d8-4c47-968c-f1a447688233'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:50.678232 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ec9c418a-e0d8-4c47-968c-f1a447688233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de52a59737cf465489cd0b47a464a1d3', 'request_id': 'req-ec9c418a-e0d8-4c47-968c-f1a447688233'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:50.685521 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ec9c418a-e0d8-4c47-968c-f1a447688233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de52a59737cf465489cd0b47a464a1d3', 'request_id': 'req-ec9c418a-e0d8-4c47-968c-f1a447688233'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:50.696848 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ec9c418a-e0d8-4c47-968c-f1a447688233 admin admin] The token cache is being invalidate because project de52a59737cf465489cd0b47a464a1d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:38:50.697154 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ec9c418a-e0d8-4c47-968c-f1a447688233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:50.697803 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-ec9c418a-e0d8-4c47-968c-f1a447688233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:50.699357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1441/2833] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:38:50 2022] DELETE /identity/v3/projects/de52a59737cf465489cd0b47a464a1d3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:50.704189 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83003fbc-b288-4c6f-8f48-92a4a08f51ef None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:50.704478 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83003fbc-b288-4c6f-8f48-92a4a08f51ef None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:50.704754 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83003fbc-b288-4c6f-8f48-92a4a08f51ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:50.722684 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-83003fbc-b288-4c6f-8f48-92a4a08f51ef None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:50.723898 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-83003fbc-b288-4c6f-8f48-92a4a08f51ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:50.781621 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1393/2834] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:50.789308 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c7304-a249-44d2-b915-a310fab64c25 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:50.789598 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c7304-a249-44d2-b915-a310fab64c25 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:50.789875 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c7304-a249-44d2-b915-a310fab64c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:50.807661 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-472c7304-a249-44d2-b915-a310fab64c25 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:50.808878 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-472c7304-a249-44d2-b915-a310fab64c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:50.817246 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1442/2835] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:50.821126 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83003fbc-b288-4c6f-8f48-92a4a08f51ef None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:50.830993 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83003fbc-b288-4c6f-8f48-92a4a08f51ef None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:50.835659 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f73ea-90c7-4901-be4d-10368ef8a4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:50.836307 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f73ea-90c7-4901-be4d-10368ef8a4cb admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:50.836591 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f73ea-90c7-4901-be4d-10368ef8a4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:50.836875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f73ea-90c7-4901-be4d-10368ef8a4cb admin admin] PATH_INFO: `/v3/domains` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:50.837593 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f73ea-90c7-4901-be4d-10368ef8a4cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 18:38:50.837900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f73ea-90c7-4901-be4d-10368ef8a4cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:50.839525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f73ea-90c7-4901-be4d-10368ef8a4cb admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:50.850887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1394/2836] 10.222.0.25 () {58 vars in 1152 bytes} [Mon May 9 18:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 18:38:50.854800 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c7304-a249-44d2-b915-a310fab64c25 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:50.863766 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c7304-a249-44d2-b915-a310fab64c25 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:50.870645 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee90be-3682-4b23-aa56-faeba2a9864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:50.871906 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee90be-3682-4b23-aa56-faeba2a9864c admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:50.872315 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee90be-3682-4b23-aa56-faeba2a9864c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:50.872598 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee90be-3682-4b23-aa56-faeba2a9864c admin admin] PATH_INFO: `/v3/projects` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:50.873286 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee90be-3682-4b23-aa56-faeba2a9864c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:50.875601 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee90be-3682-4b23-aa56-faeba2a9864c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:50.902268 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1443/2837] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:38:50 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:38:50.906347 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f73ea-90c7-4901-be4d-10368ef8a4cb admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:50.916313 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f73ea-90c7-4901-be4d-10368ef8a4cb admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:50.971408 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad43b8d-6d0a-489a-9b39-238ad22321d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:50.972088 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad43b8d-6d0a-489a-9b39-238ad22321d4 admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:50.972373 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad43b8d-6d0a-489a-9b39-238ad22321d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:50.972651 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad43b8d-6d0a-489a-9b39-238ad22321d4 admin admin] PATH_INFO: `/v3/users` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:50.973344 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad43b8d-6d0a-489a-9b39-238ad22321d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:50.975268 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad43b8d-6d0a-489a-9b39-238ad22321d4 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:50.995395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1395/2838] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:38:50 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:38:50.999208 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee90be-3682-4b23-aa56-faeba2a9864c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:51.009972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee90be-3682-4b23-aa56-faeba2a9864c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:51.014571 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb49d7-a094-4b13-a880-7e39b33db9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:51.015321 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb49d7-a094-4b13-a880-7e39b33db9ad admin admin] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:51.015636 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb49d7-a094-4b13-a880-7e39b33db9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:51.016127 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb49d7-a094-4b13-a880-7e39b33db9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:51.016850 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb49d7-a094-4b13-a880-7e39b33db9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:51.018570 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb49d7-a094-4b13-a880-7e39b33db9ad admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:51.029409 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1444/2839] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:38:51.033150 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad43b8d-6d0a-489a-9b39-238ad22321d4 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:51.043425 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad43b8d-6d0a-489a-9b39-238ad22321d4 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:51.048202 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151f3cc-28c3-4110-862d-c49a2188e2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:51.048870 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151f3cc-28c3-4110-862d-c49a2188e2ff admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:51.049668 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151f3cc-28c3-4110-862d-c49a2188e2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:51.049945 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151f3cc-28c3-4110-862d-c49a2188e2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:51.050700 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151f3cc-28c3-4110-862d-c49a2188e2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:51.052316 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151f3cc-28c3-4110-862d-c49a2188e2ff admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:51.066512 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1396/2840] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:38:51.070198 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb49d7-a094-4b13-a880-7e39b33db9ad admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:51.080360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb49d7-a094-4b13-a880-7e39b33db9ad admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:51.085092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d3242-9afc-4dfb-bb86-7596d651fef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:51.085969 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d3242-9afc-4dfb-bb86-7596d651fef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:51.086248 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d3242-9afc-4dfb-bb86-7596d651fef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:51.086524 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d3242-9afc-4dfb-bb86-7596d651fef3 admin admin] PATH_INFO: `/v3/projects/93b84cf540d348be994e2ddab8345520/users/0784cab1d4c142d7aeb8557ad2ab146c/roles/e1dddf0b88ee4578822ce51a2aba6760` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:51.101225 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d3242-9afc-4dfb-bb86-7596d651fef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93b84cf540d348be994e2ddab8345520, user_id=0784cab1d4c142d7aeb8557ad2ab146c, role_id=e1dddf0b88ee4578822ce51a2aba6760)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:51.103842 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d3242-9afc-4dfb-bb86-7596d651fef3 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:51.115886 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151f3cc-28c3-4110-862d-c49a2188e2ff admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:51.117678 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-4151f3cc-28c3-4110-862d-c49a2188e2ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:51.118060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1445/2841] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:38:51 2022] PUT /identity/v3/projects/93b84cf540d348be994e2ddab8345520/users/0784cab1d4c142d7aeb8557ad2ab146c/roles/e1dddf0b88ee4578822ce51a2aba6760 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:51.123082 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc5417-c247-4def-ac27-0f1dd9402a0e None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:51.123397 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc5417-c247-4def-ac27-0f1dd9402a0e None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:51.123674 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc5417-c247-4def-ac27-0f1dd9402a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:51.133890 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151f3cc-28c3-4110-862d-c49a2188e2ff admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:51.141120 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-6dfc5417-c247-4def-ac27-0f1dd9402a0e None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:51.142384 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-6dfc5417-c247-4def-ac27-0f1dd9402a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:51.193204 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73810a5b-ffa3-41cf-be08-fbe82ff10f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:51.194097 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73810a5b-ffa3-41cf-be08-fbe82ff10f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:51.194380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73810a5b-ffa3-41cf-be08-fbe82ff10f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:51.194661 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73810a5b-ffa3-41cf-be08-fbe82ff10f6f admin admin] PATH_INFO: `/v3/users/bf8348f7d99b4fdb892d25571a6d1202` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:51.195858 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73810a5b-ffa3-41cf-be08-fbe82ff10f6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf8348f7d99b4fdb892d25571a6d1202)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:51.197763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73810a5b-ffa3-41cf-be08-fbe82ff10f6f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:51.199360 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1446/2842] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:38:51.208789 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc5417-c247-4def-ac27-0f1dd9402a0e None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:51.210166 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-6dfc5417-c247-4def-ac27-0f1dd9402a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:51.247226 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-73810a5b-ffa3-41cf-be08-fbe82ff10f6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf8348f7d99b4fdb892d25571a6d1202'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:51.254731 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-73810a5b-ffa3-41cf-be08-fbe82ff10f6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf8348f7d99b4fdb892d25571a6d1202'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:51.255459 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-73810a5b-ffa3-41cf-be08-fbe82ff10f6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf8348f7d99b4fdb892d25571a6d1202'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:51.263161 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-73810a5b-ffa3-41cf-be08-fbe82ff10f6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf8348f7d99b4fdb892d25571a6d1202'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:51.263840 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-73810a5b-ffa3-41cf-be08-fbe82ff10f6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf8348f7d99b4fdb892d25571a6d1202'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:51.271440 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc5417-c247-4def-ac27-0f1dd9402a0e None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:51.275038 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c942fad-d72e-430c-82ae-1ed582015fd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:51.275713 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942fad-d72e-430c-82ae-1ed582015fd8 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:51.275997 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942fad-d72e-430c-82ae-1ed582015fd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:51.276272 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942fad-d72e-430c-82ae-1ed582015fd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:51.278014 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9c942fad-d72e-430c-82ae-1ed582015fd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:51.281336 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1397/2843] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:38:51 2022] DELETE /identity/v3/users/bf8348f7d99b4fdb892d25571a6d1202 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:51.285460 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649196d-eec2-4623-a279-9f048fe23ce3 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:51.285815 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649196d-eec2-4623-a279-9f048fe23ce3 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:51.286178 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649196d-eec2-4623-a279-9f048fe23ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:51.291874 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c942fad-d72e-430c-82ae-1ed582015fd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:51.293306 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c942fad-d72e-430c-82ae-1ed582015fd8 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:51.303165 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1447/2844] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:38:51.303561 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-9649196d-eec2-4623-a279-9f048fe23ce3 None None] MFA Rules not processed for user `a379c334e6cd446c80b18fda6ce9162e`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:51.305429 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9649196d-eec2-4623-a279-9f048fe23ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:51.365998 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1398/2845] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:38:51.376112 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c942fad-d72e-430c-82ae-1ed582015fd8 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:51.377763 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-9c942fad-d72e-430c-82ae-1ed582015fd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:51.388761 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c942fad-d72e-430c-82ae-1ed582015fd8 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:51.439376 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf98dd-c4ba-4329-8e91-9819b45d4b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:51.440053 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf98dd-c4ba-4329-8e91-9819b45d4b86 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:51.440448 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf98dd-c4ba-4329-8e91-9819b45d4b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:51.440723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf98dd-c4ba-4329-8e91-9819b45d4b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:51.450602 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf98dd-c4ba-4329-8e91-9819b45d4b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:51.452242 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf98dd-c4ba-4329-8e91-9819b45d4b86 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:51.462896 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1448/2846] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:38:51.865237 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649196d-eec2-4623-a279-9f048fe23ce3 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:51.866715 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9649196d-eec2-4623-a279-9f048fe23ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:51.879095 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649196d-eec2-4623-a279-9f048fe23ce3 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:51.884413 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:51.884967 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:51.885340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:51.885673 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] PATH_INFO: `/v3/projects/7d186503e14a4674bebd028d10b3726b` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:51.886915 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d186503e14a4674bebd028d10b3726b)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:51.889085 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:51.937579 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d186503e14a4674bebd028d10b3726b', 'request_id': 'req-7c8a7412-2ee0-4614-8008-d76bea60ff25'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:51.943291 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d186503e14a4674bebd028d10b3726b', 'request_id': 'req-7c8a7412-2ee0-4614-8008-d76bea60ff25'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:51.951496 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d186503e14a4674bebd028d10b3726b', 'request_id': 'req-7c8a7412-2ee0-4614-8008-d76bea60ff25'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:51.964245 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] The token cache is being invalidate because project 7d186503e14a4674bebd028d10b3726b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:38:51.965210 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:51.966081 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:38:51.967932 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1399/2847] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:38:51 2022] DELETE /identity/v3/projects/7d186503e14a4674bebd028d10b3726b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:55.160656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2d8e0-6c12-4ed3-9f32-0faf2a3dc26f None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:55.160656 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2d8e0-6c12-4ed3-9f32-0faf2a3dc26f None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:55.161259 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2d8e0-6c12-4ed3-9f32-0faf2a3dc26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:55.190948 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-07c2d8e0-6c12-4ed3-9f32-0faf2a3dc26f None None] MFA Rules not processed for user `0784cab1d4c142d7aeb8557ad2ab146c`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:38:55.192254 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-07c2d8e0-6c12-4ed3-9f32-0faf2a3dc26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:55.257000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1449/2848] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:38:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:38:55.261534 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:55.262951 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:38:55.326376 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a7412-2ee0-4614-8008-d76bea60ff25 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:55.330976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aadb1-aa91-4459-a75a-3d8176bab04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:55.331687 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aadb1-aa91-4459-a75a-3d8176bab04f admin admin] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:55.332018 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aadb1-aa91-4459-a75a-3d8176bab04f admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:55.332299 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aadb1-aa91-4459-a75a-3d8176bab04f admin admin] PATH_INFO: `/v3/projects` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:55.332984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aadb1-aa91-4459-a75a-3d8176bab04f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:55.334869 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aadb1-aa91-4459-a75a-3d8176bab04f admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:55.363210 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1400/2849] 10.222.0.25 () {60 vars in 1152 bytes} [Mon May 9 18:38:55 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 18:38:55.367263 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2d8e0-6c12-4ed3-9f32-0faf2a3dc26f None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:55.378716 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2d8e0-6c12-4ed3-9f32-0faf2a3dc26f None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:55.434408 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1677d7-b7d5-469c-9b3f-6d9f33b4e419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:55.435100 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1677d7-b7d5-469c-9b3f-6d9f33b4e419 admin admin] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:55.435421 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1677d7-b7d5-469c-9b3f-6d9f33b4e419 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:55.435698 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1677d7-b7d5-469c-9b3f-6d9f33b4e419 admin admin] PATH_INFO: `/v3/users` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:55.436422 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1677d7-b7d5-469c-9b3f-6d9f33b4e419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:55.438639 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1677d7-b7d5-469c-9b3f-6d9f33b4e419 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:55.462849 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1450/2850] 10.222.0.25 () {60 vars in 1143 bytes} [Mon May 9 18:38:55 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 18:38:55.464066 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aadb1-aa91-4459-a75a-3d8176bab04f admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:55.475614 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aadb1-aa91-4459-a75a-3d8176bab04f admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:55.483770 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c1043-22dd-4109-8af4-d70bc1c458e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:55.484521 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c1043-22dd-4109-8af4-d70bc1c458e9 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:55.484873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c1043-22dd-4109-8af4-d70bc1c458e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:55.485220 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c1043-22dd-4109-8af4-d70bc1c458e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:55.486039 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c1043-22dd-4109-8af4-d70bc1c458e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:55.487984 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c1043-22dd-4109-8af4-d70bc1c458e9 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:55.505174 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1401/2851] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:38:55.509538 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1677d7-b7d5-469c-9b3f-6d9f33b4e419 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:55.522112 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1677d7-b7d5-469c-9b3f-6d9f33b4e419 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:55.526676 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8eead-5f1c-4319-bae6-87b89e66f90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:55.527599 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8eead-5f1c-4319-bae6-87b89e66f90c admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:55.527876 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8eead-5f1c-4319-bae6-87b89e66f90c admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:55.528149 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8eead-5f1c-4319-bae6-87b89e66f90c admin admin] PATH_INFO: `/v3/projects/160d33e6e5b1470c9de7b0cf18029a59/users/939eb1f1c6a246c3a13ee59cfde9ab1f/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:55.547430 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8eead-5f1c-4319-bae6-87b89e66f90c admin admin] RBAC: Authorizing `identity:create_grant(project_id=160d33e6e5b1470c9de7b0cf18029a59, user_id=939eb1f1c6a246c3a13ee59cfde9ab1f, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:55.554627 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8eead-5f1c-4319-bae6-87b89e66f90c admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:55.572737 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1451/2852] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:38:55 2022] PUT /identity/v3/projects/160d33e6e5b1470c9de7b0cf18029a59/users/939eb1f1c6a246c3a13ee59cfde9ab1f/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:38:55.577184 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c1043-22dd-4109-8af4-d70bc1c458e9 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:55.587916 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c1043-22dd-4109-8af4-d70bc1c458e9 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:55.646079 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69645ca-4536-4b4f-97c6-9a0490e127c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:55.647132 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69645ca-4536-4b4f-97c6-9a0490e127c3 admin admin] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:55.647514 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69645ca-4536-4b4f-97c6-9a0490e127c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:55.647789 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69645ca-4536-4b4f-97c6-9a0490e127c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:55.648509 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69645ca-4536-4b4f-97c6-9a0490e127c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:55.650086 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69645ca-4536-4b4f-97c6-9a0490e127c3 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:55.661005 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1402/2853] 10.222.0.25 () {58 vars in 1121 bytes} [Mon May 9 18:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 18:38:55.665156 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8eead-5f1c-4319-bae6-87b89e66f90c admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:38:55.676614 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8eead-5f1c-4319-bae6-87b89e66f90c admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:38:55.681796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564fd13-c76e-4d70-bfbf-7057911fc623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:38:55.682773 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564fd13-c76e-4d70-bfbf-7057911fc623 admin admin] REQUEST_METHOD: `PUT` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:38:55.683125 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564fd13-c76e-4d70-bfbf-7057911fc623 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:38:55.683583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564fd13-c76e-4d70-bfbf-7057911fc623 admin admin] PATH_INFO: `/v3/projects/160d33e6e5b1470c9de7b0cf18029a59/users/939eb1f1c6a246c3a13ee59cfde9ab1f/roles/ee562e637fc64608b9129ddd09ec0cdc` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:38:55.685583 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564fd13-c76e-4d70-bfbf-7057911fc623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=160d33e6e5b1470c9de7b0cf18029a59, user_id=939eb1f1c6a246c3a13ee59cfde9ab1f, role_id=ee562e637fc64608b9129ddd09ec0cdc)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:38:55.688281 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564fd13-c76e-4d70-bfbf-7057911fc623 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:38:55.699705 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1452/2854] 10.222.0.25 () {60 vars in 1482 bytes} [Mon May 9 18:38:55 2022] PUT /identity/v3/projects/160d33e6e5b1470c9de7b0cf18029a59/users/939eb1f1c6a246c3a13ee59cfde9ab1f/roles/ee562e637fc64608b9129ddd09ec0cdc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:39:00.372391 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3cb19-e92c-4b7d-861c-64dd4d55a3b3 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:39:00.372898 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3cb19-e92c-4b7d-861c-64dd4d55a3b3 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:39:00.373005 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3cb19-e92c-4b7d-861c-64dd4d55a3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:39:00.399891 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-ecc3cb19-e92c-4b7d-861c-64dd4d55a3b3 None None] MFA Rules not processed for user `939eb1f1c6a246c3a13ee59cfde9ab1f`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:39:00.401135 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-ecc3cb19-e92c-4b7d-861c-64dd4d55a3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:39:00.468160 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1403/2855] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:39:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:39:00.480563 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564fd13-c76e-4d70-bfbf-7057911fc623 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:39:00.482241 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-c564fd13-c76e-4d70-bfbf-7057911fc623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:39:00.537912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564fd13-c76e-4d70-bfbf-7057911fc623 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:39:00.542000 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5ddb7-73e3-406f-a072-64728bb0cbe6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cf284e81ba4042b66c0bc189f1ae2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:39:00.542655 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5ddb7-73e3-406f-a072-64728bb0cbe6 service glance] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:39:00.542938 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5ddb7-73e3-406f-a072-64728bb0cbe6 service glance] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:39:00.543236 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5ddb7-73e3-406f-a072-64728bb0cbe6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:39:00.601136 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5ddb7-73e3-406f-a072-64728bb0cbe6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:39:00.602646 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5ddb7-73e3-406f-a072-64728bb0cbe6 service glance] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:39:00.612600 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1453/2856] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:39:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:39:00.718091 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e333e-0bb5-4f8f-9545-a7823b04eed1 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:39:00.718634 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e333e-0bb5-4f8f-9545-a7823b04eed1 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:39:00.718853 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e333e-0bb5-4f8f-9545-a7823b04eed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:39:00.742972 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-402e333e-0bb5-4f8f-9545-a7823b04eed1 None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:39:00.744525 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-402e333e-0bb5-4f8f-9545-a7823b04eed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:39:00.805252 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1404/2857] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:39:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:39:00.817764 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5ddb7-73e3-406f-a072-64728bb0cbe6 service glance] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:39:00.819140 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-89b5ddb7-73e3-406f-a072-64728bb0cbe6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:39:00.888658 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5ddb7-73e3-406f-a072-64728bb0cbe6 service glance] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:39:00.895746 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50292250-53b9-4ae7-b5b7-9819f3ab22ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:39:00.896509 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50292250-53b9-4ae7-b5b7-9819f3ab22ac service swift] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:39:00.896912 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50292250-53b9-4ae7-b5b7-9819f3ab22ac service swift] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:39:00.897192 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50292250-53b9-4ae7-b5b7-9819f3ab22ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:39:00.913294 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50292250-53b9-4ae7-b5b7-9819f3ab22ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:39:00.915266 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50292250-53b9-4ae7-b5b7-9819f3ab22ac service swift] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:39:00.925841 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1454/2858] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:39:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 18:39:09.601942 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d164fda-5e6b-4cfb-b411-0b0ff8fe5303 None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:39:09.602425 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d164fda-5e6b-4cfb-b411-0b0ff8fe5303 None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:39:09.602552 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d164fda-5e6b-4cfb-b411-0b0ff8fe5303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:39:09.620759 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-9d164fda-5e6b-4cfb-b411-0b0ff8fe5303 None None] MFA Rules not processed for user `0784cab1d4c142d7aeb8557ad2ab146c`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:39:09.621961 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9d164fda-5e6b-4cfb-b411-0b0ff8fe5303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:39:09.628887 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1405/2859] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:39:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:39:11.380007 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d74ced-dc14-4ca4-abed-b645a223ef8d None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:39:11.380543 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d74ced-dc14-4ca4-abed-b645a223ef8d None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:39:11.380778 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d74ced-dc14-4ca4-abed-b645a223ef8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:39:11.403387 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-32d74ced-dc14-4ca4-abed-b645a223ef8d None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:39:11.404744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-32d74ced-dc14-4ca4-abed-b645a223ef8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:39:11.414152 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1455/2860] 10.222.0.25 () {58 vars in 1009 bytes} [Mon May 9 18:39:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 18:39:11.429256 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d164fda-5e6b-4cfb-b411-0b0ff8fe5303 None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:39:11.442170 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d164fda-5e6b-4cfb-b411-0b0ff8fe5303 None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:39:11.447900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cbac5-f6e7-4fd4-8c4e-b9cbfcda0f37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:39:11.448696 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cbac5-f6e7-4fd4-8c4e-b9cbfcda0f37 service swift] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:39:11.449035 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cbac5-f6e7-4fd4-8c4e-b9cbfcda0f37 service swift] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:39:11.449374 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cbac5-f6e7-4fd4-8c4e-b9cbfcda0f37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:39:11.458045 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cbac5-f6e7-4fd4-8c4e-b9cbfcda0f37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:39:11.459744 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cbac5-f6e7-4fd4-8c4e-b9cbfcda0f37 service swift] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:39:11.467954 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1406/2861] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:39:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 18:39:37.830157 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d74ced-dc14-4ca4-abed-b645a223ef8d None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:39:37.831663 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-32d74ced-dc14-4ca4-abed-b645a223ef8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:39:37.888696 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d74ced-dc14-4ca4-abed-b645a223ef8d None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:39:37.892129 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd4473-1f9a-4c91-abeb-05cc8d1a83ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:39:37.892773 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd4473-1f9a-4c91-abeb-05cc8d1a83ea service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:39:37.893041 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd4473-1f9a-4c91-abeb-05cc8d1a83ea service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:39:37.893310 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd4473-1f9a-4c91-abeb-05cc8d1a83ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:39:37.895025 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-f1fd4473-1f9a-4c91-abeb-05cc8d1a83ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:39:37.961327 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd4473-1f9a-4c91-abeb-05cc8d1a83ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:39:37.962750 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd4473-1f9a-4c91-abeb-05cc8d1a83ea service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:39:37.974285 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1456/2862] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:39:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:39:38.097249 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cbac5-f6e7-4fd4-8c4e-b9cbfcda0f37 service swift] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:39:38.106876 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cbac5-f6e7-4fd4-8c4e-b9cbfcda0f37 service swift] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:39:38.110422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f94fc-9ecd-49c3-8fd2-bbbcbc2a2da1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:39:38.111053 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f94fc-9ecd-49c3-8fd2-bbbcbc2a2da1 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:39:38.111377 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f94fc-9ecd-49c3-8fd2-bbbcbc2a2da1 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:39:38.111656 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f94fc-9ecd-49c3-8fd2-bbbcbc2a2da1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:39:38.121239 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f94fc-9ecd-49c3-8fd2-bbbcbc2a2da1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:39:38.122648 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f94fc-9ecd-49c3-8fd2-bbbcbc2a2da1 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:39:38.131982 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1407/2863] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:39:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:39:41.553400 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd4473-1f9a-4c91-abeb-05cc8d1a83ea service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:39:41.563012 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd4473-1f9a-4c91-abeb-05cc8d1a83ea service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:39:41.567723 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6443c7c-0160-485b-b6d1-2337599af6aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:39:41.568624 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6443c7c-0160-485b-b6d1-2337599af6aa service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:39:41.569060 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6443c7c-0160-485b-b6d1-2337599af6aa service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:39:41.569501 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6443c7c-0160-485b-b6d1-2337599af6aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:39:41.571556 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e6443c7c-0160-485b-b6d1-2337599af6aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:39:41.594183 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6443c7c-0160-485b-b6d1-2337599af6aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:39:41.595816 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6443c7c-0160-485b-b6d1-2337599af6aa service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:39:41.608774 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1457/2864] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:39:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 18:40:04.289877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f94fc-9ecd-49c3-8fd2-bbbcbc2a2da1 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:40:04.298636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f94fc-9ecd-49c3-8fd2-bbbcbc2a2da1 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:40:04.302378 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fd59e-45f5-468d-9e45-d9507f162dd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:40:04.303046 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fd59e-45f5-468d-9e45-d9507f162dd7 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:40:04.303373 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fd59e-45f5-468d-9e45-d9507f162dd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:40:04.303653 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fd59e-45f5-468d-9e45-d9507f162dd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:40:04.305404 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-364fd59e-45f5-468d-9e45-d9507f162dd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:40:04.315323 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fd59e-45f5-468d-9e45-d9507f162dd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:40:04.316971 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fd59e-45f5-468d-9e45-d9507f162dd7 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:40:04.325545 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1408/2865] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:40:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:40:07.443978 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6bbff-723a-4e37-aea2-c3db7d4d999c None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:40:07.444505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6bbff-723a-4e37-aea2-c3db7d4d999c None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:40:07.444505 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6bbff-723a-4e37-aea2-c3db7d4d999c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:40:07.461916 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-43e6bbff-723a-4e37-aea2-c3db7d4d999c None None] MFA Rules not processed for user `d1934a2e2edb4b8a93baac501c9c5c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:40:07.463092 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-43e6bbff-723a-4e37-aea2-c3db7d4d999c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:40:07.514375 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1458/2866] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:40:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:40:08.261840 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fd59e-45f5-468d-9e45-d9507f162dd7 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:40:08.263746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-364fd59e-45f5-468d-9e45-d9507f162dd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:40:08.331111 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fd59e-45f5-468d-9e45-d9507f162dd7 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:40:08.336608 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917a028-d3ee-4719-991e-e45d853d3b0a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f747dcb955fa499a972c46587d3db294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:40:08.337260 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917a028-d3ee-4719-991e-e45d853d3b0a service placement] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:40:08.337538 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917a028-d3ee-4719-991e-e45d853d3b0a service placement] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:40:08.337810 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917a028-d3ee-4719-991e-e45d853d3b0a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:40:08.339535 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-8917a028-d3ee-4719-991e-e45d853d3b0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:40:08.355016 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917a028-d3ee-4719-991e-e45d853d3b0a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:40:08.356587 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917a028-d3ee-4719-991e-e45d853d3b0a service placement] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:40:08.366195 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1409/2867] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:40:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:40:10.918818 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6bbff-723a-4e37-aea2-c3db7d4d999c None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:40:10.921822 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-43e6bbff-723a-4e37-aea2-c3db7d4d999c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:40:10.939963 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6bbff-723a-4e37-aea2-c3db7d4d999c None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:40:10.946801 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9cc82-d255-4485-bdd0-7dea39043aa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:40:10.947495 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9cc82-d255-4485-bdd0-7dea39043aa8 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:40:10.947780 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9cc82-d255-4485-bdd0-7dea39043aa8 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:40:10.948054 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9cc82-d255-4485-bdd0-7dea39043aa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:40:10.949872 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-95b9cc82-d255-4485-bdd0-7dea39043aa8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:40:10.962883 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9cc82-d255-4485-bdd0-7dea39043aa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:40:10.964304 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9cc82-d255-4485-bdd0-7dea39043aa8 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:40:10.976255 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1459/2868] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:40:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:41:04.548103 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91400f-171b-4564-ae4e-a97a36977dcf None None] REQUEST_METHOD: `POST` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:41:04.548607 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91400f-171b-4564-ae4e-a97a36977dcf None None] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:41:04.548718 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91400f-171b-4564-ae4e-a97a36977dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:41:04.565746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.auth.core [None req-df91400f-171b-4564-ae4e-a97a36977dcf None None] MFA Rules not processed for user `154f86b7575e47efb2a5e1c35943e037`. Rule list: `[]` (Enabled: `True`). {{(pid=79825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:41:04.566901 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-df91400f-171b-4564-ae4e-a97a36977dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:41:04.574418 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1410/2869] 10.222.0.25 () {58 vars in 967 bytes} [Mon May 9 18:41:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:41:04.613265 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9cc82-d255-4485-bdd0-7dea39043aa8 service nova] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:41:04.624731 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9cc82-d255-4485-bdd0-7dea39043aa8 service nova] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:41:04.629638 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79372c-19a2-4f16-b301-0df596e26285 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec2db3aa2b7d41f89314fec960d898b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:41:04.630266 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79372c-19a2-4f16-b301-0df596e26285 service swift] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:41:04.630548 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79372c-19a2-4f16-b301-0df596e26285 service swift] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:41:04.630836 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79372c-19a2-4f16-b301-0df596e26285 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:41:04.639070 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79372c-19a2-4f16-b301-0df596e26285 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:41:04.640644 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79372c-19a2-4f16-b301-0df596e26285 service swift] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:41:04.648917 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1460/2870] 10.222.0.25 () {60 vars in 1384 bytes} [Mon May 9 18:41:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 18:41:11.292544 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91400f-171b-4564-ae4e-a97a36977dcf None None] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:41:11.354370 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91400f-171b-4564-ae4e-a97a36977dcf None None] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:41:11.358788 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e357724-0341-4554-9b2f-ab7ea2acbc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:41:11.359697 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e357724-0341-4554-9b2f-ab7ea2acbc6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:41:11.359976 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e357724-0341-4554-9b2f-ab7ea2acbc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:41:11.360242 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e357724-0341-4554-9b2f-ab7ea2acbc6e admin admin] PATH_INFO: `/v3/users/0784cab1d4c142d7aeb8557ad2ab146c` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:41:11.361395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e357724-0341-4554-9b2f-ab7ea2acbc6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0784cab1d4c142d7aeb8557ad2ab146c)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:41:11.363373 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e357724-0341-4554-9b2f-ab7ea2acbc6e admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:41:11.412110 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1e357724-0341-4554-9b2f-ab7ea2acbc6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0784cab1d4c142d7aeb8557ad2ab146c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.420340 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1e357724-0341-4554-9b2f-ab7ea2acbc6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0784cab1d4c142d7aeb8557ad2ab146c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.421022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1e357724-0341-4554-9b2f-ab7ea2acbc6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0784cab1d4c142d7aeb8557ad2ab146c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.427633 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1e357724-0341-4554-9b2f-ab7ea2acbc6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0784cab1d4c142d7aeb8557ad2ab146c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.428789 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-1e357724-0341-4554-9b2f-ab7ea2acbc6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0784cab1d4c142d7aeb8557ad2ab146c'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.445362 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1411/2871] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:41:11 2022] DELETE /identity/v3/users/0784cab1d4c142d7aeb8557ad2ab146c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:41:11.611419 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79372c-19a2-4f16-b301-0df596e26285 service swift] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:41:11.621823 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79372c-19a2-4f16-b301-0df596e26285 service swift] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:41:11.677976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:41:11.678781 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:41:11.679066 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:41:11.679373 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] PATH_INFO: `/v3/projects/93b84cf540d348be994e2ddab8345520` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:41:11.680547 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93b84cf540d348be994e2ddab8345520)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:41:11.682432 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:41:11.727434 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93b84cf540d348be994e2ddab8345520', 'request_id': 'req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.732877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93b84cf540d348be994e2ddab8345520', 'request_id': 'req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.739591 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93b84cf540d348be994e2ddab8345520', 'request_id': 'req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.750084 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] The token cache is being invalidate because project 93b84cf540d348be994e2ddab8345520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:41:11.750374 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.750992 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.752526 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1461/2872] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:41:11 2022] DELETE /identity/v3/projects/93b84cf540d348be994e2ddab8345520 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:41:11.756498 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e357724-0341-4554-9b2f-ab7ea2acbc6e admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:41:11.757880 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1e357724-0341-4554-9b2f-ab7ea2acbc6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:41:11.770580 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e357724-0341-4554-9b2f-ab7ea2acbc6e admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:41:11.824729 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2bf82-9918-469b-be67-1a15c976111c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:41:11.825595 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2bf82-9918-469b-be67-1a15c976111c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:41:11.825873 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2bf82-9918-469b-be67-1a15c976111c admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:41:11.826145 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2bf82-9918-469b-be67-1a15c976111c admin admin] PATH_INFO: `/v3/users/939eb1f1c6a246c3a13ee59cfde9ab1f` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:41:11.827394 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2bf82-9918-469b-be67-1a15c976111c admin admin] RBAC: Authorizing `identity:delete_user(user_id=939eb1f1c6a246c3a13ee59cfde9ab1f)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:41:11.829445 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2bf82-9918-469b-be67-1a15c976111c admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:41:11.878638 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-46e2bf82-9918-469b-be67-1a15c976111c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939eb1f1c6a246c3a13ee59cfde9ab1f'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.885877 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-46e2bf82-9918-469b-be67-1a15c976111c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939eb1f1c6a246c3a13ee59cfde9ab1f'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.886536 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-46e2bf82-9918-469b-be67-1a15c976111c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939eb1f1c6a246c3a13ee59cfde9ab1f'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.893296 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-46e2bf82-9918-469b-be67-1a15c976111c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939eb1f1c6a246c3a13ee59cfde9ab1f'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.893894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-46e2bf82-9918-469b-be67-1a15c976111c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939eb1f1c6a246c3a13ee59cfde9ab1f'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:11.910636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1412/2873] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:41:11 2022] DELETE /identity/v3/users/939eb1f1c6a246c3a13ee59cfde9ab1f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:41:12.085172 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:41:12.086609 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:41:12.101270 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1864c85-4b7b-4e63-90f0-4ae9037b9ce6 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:41:12.165150 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70e601-a70b-4712-9b1c-a6e078d572e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:41:12.166034 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70e601-a70b-4712-9b1c-a6e078d572e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:41:12.166316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70e601-a70b-4712-9b1c-a6e078d572e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:41:12.166590 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70e601-a70b-4712-9b1c-a6e078d572e4 admin admin] PATH_INFO: `/v3/projects/160d33e6e5b1470c9de7b0cf18029a59` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:41:12.167817 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70e601-a70b-4712-9b1c-a6e078d572e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=160d33e6e5b1470c9de7b0cf18029a59)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:41:12.169712 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70e601-a70b-4712-9b1c-a6e078d572e4 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:41:12.217448 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9c70e601-a70b-4712-9b1c-a6e078d572e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '160d33e6e5b1470c9de7b0cf18029a59', 'request_id': 'req-9c70e601-a70b-4712-9b1c-a6e078d572e4'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:12.223531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9c70e601-a70b-4712-9b1c-a6e078d572e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '160d33e6e5b1470c9de7b0cf18029a59', 'request_id': 'req-9c70e601-a70b-4712-9b1c-a6e078d572e4'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:12.230779 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9c70e601-a70b-4712-9b1c-a6e078d572e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '160d33e6e5b1470c9de7b0cf18029a59', 'request_id': 'req-9c70e601-a70b-4712-9b1c-a6e078d572e4'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:12.241846 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9c70e601-a70b-4712-9b1c-a6e078d572e4 admin admin] The token cache is being invalidate because project 160d33e6e5b1470c9de7b0cf18029a59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:41:12.242133 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9c70e601-a70b-4712-9b1c-a6e078d572e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:12.242769 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-9c70e601-a70b-4712-9b1c-a6e078d572e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:41:12.244704 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1462/2874] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:41:12 2022] DELETE /identity/v3/projects/160d33e6e5b1470c9de7b0cf18029a59 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:42:03.703594 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2bf82-9918-469b-be67-1a15c976111c admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:42:03.704981 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-46e2bf82-9918-469b-be67-1a15c976111c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:42:03.787049 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2bf82-9918-469b-be67-1a15c976111c admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:42:03.833136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f2f2c-b6e8-4dc7-a98f-4a0628fc443f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bda352a10f241ca96d5308a55238d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:42:03.833782 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f2f2c-b6e8-4dc7-a98f-4a0628fc443f service cinder] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:42:03.834063 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f2f2c-b6e8-4dc7-a98f-4a0628fc443f service cinder] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:42:03.834337 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f2f2c-b6e8-4dc7-a98f-4a0628fc443f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:42:03.836125 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-3a1f2f2c-b6e8-4dc7-a98f-4a0628fc443f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:42:03.904222 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f2f2c-b6e8-4dc7-a98f-4a0628fc443f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:42:03.905635 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f2f2c-b6e8-4dc7-a98f-4a0628fc443f service cinder] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:42:03.915434 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1413/2875] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:42:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:42:10.684776 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e00636-c919-4de2-8169-df816dd3c373 None None] REQUEST_METHOD: `POST` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:42:10.685321 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e00636-c919-4de2-8169-df816dd3c373 None None] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:42:10.685321 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e00636-c919-4de2-8169-df816dd3c373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:42:10.702412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.auth.core [None req-a2e00636-c919-4de2-8169-df816dd3c373 None None] MFA Rules not processed for user `d1934a2e2edb4b8a93baac501c9c5c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=79826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 18:42:10.703613 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a2e00636-c919-4de2-8169-df816dd3c373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:42:10.710288 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1463/2876] 10.222.0.25 () {58 vars in 957 bytes} [Mon May 9 18:42:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 18:42:11.014380 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f2f2c-b6e8-4dc7-a98f-4a0628fc443f service cinder] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:42:11.015776 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-3a1f2f2c-b6e8-4dc7-a98f-4a0628fc443f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:42:11.090238 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f2f2c-b6e8-4dc7-a98f-4a0628fc443f service cinder] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:42:11.093896 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db477dc7-f304-4d36-a27f-9946f3863403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:42:11.094533 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db477dc7-f304-4d36-a27f-9946f3863403 service neutron] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:42:11.094813 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db477dc7-f304-4d36-a27f-9946f3863403 service neutron] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:42:11.095083 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db477dc7-f304-4d36-a27f-9946f3863403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:42:11.096800 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-db477dc7-f304-4d36-a27f-9946f3863403 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:42:11.106746 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db477dc7-f304-4d36-a27f-9946f3863403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:42:11.108159 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db477dc7-f304-4d36-a27f-9946f3863403 service neutron] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:42:11.115900 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1414/2877] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:42:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:42:24.697251 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e00636-c919-4de2-8169-df816dd3c373 None None] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:42:24.698933 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-a2e00636-c919-4de2-8169-df816dd3c373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:42:24.776976 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e00636-c919-4de2-8169-df816dd3c373 None None] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:42:24.782356 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902f06b-1be6-420e-a93b-782f9bf39506 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f769ee9f31354e5296645fb1e3bd84b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:42:24.782999 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902f06b-1be6-420e-a93b-782f9bf39506 service nova] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:42:24.783316 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902f06b-1be6-420e-a93b-782f9bf39506 service nova] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:42:24.783593 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902f06b-1be6-420e-a93b-782f9bf39506 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:42:24.785334 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-e902f06b-1be6-420e-a93b-782f9bf39506 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:42:24.797354 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e902f06b-1be6-420e-a93b-782f9bf39506 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:42:24.798916 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e902f06b-1be6-420e-a93b-782f9bf39506 service nova] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:42:24.809714 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1464/2878] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:42:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:43:50.388894 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db477dc7-f304-4d36-a27f-9946f3863403 service neutron] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:43:50.390295 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-db477dc7-f304-4d36-a27f-9946f3863403 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:43:50.476452 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db477dc7-f304-4d36-a27f-9946f3863403 service neutron] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:43:50.482109 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43eb56-c127-48c7-979e-f9e75a01a480 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f747dcb955fa499a972c46587d3db294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:43:50.482763 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43eb56-c127-48c7-979e-f9e75a01a480 service placement] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:43:50.483047 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43eb56-c127-48c7-979e-f9e75a01a480 service placement] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:43:50.483348 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43eb56-c127-48c7-979e-f9e75a01a480 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:43:50.485090 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1d43eb56-c127-48c7-979e-f9e75a01a480 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:43:50.496184 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43eb56-c127-48c7-979e-f9e75a01a480 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:43:50.497742 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43eb56-c127-48c7-979e-f9e75a01a480 service placement] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:43:50.507636 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1415/2879] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:43:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:44:13.696436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902f06b-1be6-420e-a93b-782f9bf39506 service nova] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:44:13.706972 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902f06b-1be6-420e-a93b-782f9bf39506 service nova] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:44:13.710484 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9e500-48ad-472e-8763-808fbad47082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2b6318776b422d9701351ba6b4ee27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:44:13.711116 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9e500-48ad-472e-8763-808fbad47082 service neutron] REQUEST_METHOD: `GET` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:44:13.711464 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9e500-48ad-472e-8763-808fbad47082 service neutron] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:44:13.711744 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9e500-48ad-472e-8763-808fbad47082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:44:13.713518 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-58e9e500-48ad-472e-8763-808fbad47082 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:44:13.797006 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9e500-48ad-472e-8763-808fbad47082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:44:13.798330 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9e500-48ad-472e-8763-808fbad47082 service neutron] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:44:13.816950 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1465/2880] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:44:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 18:44:19.832965 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43eb56-c127-48c7-979e-f9e75a01a480 service placement] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:44:19.834553 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-1d43eb56-c127-48c7-979e-f9e75a01a480 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:44:19.849395 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43eb56-c127-48c7-979e-f9e75a01a480 service placement] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:44:19.853990 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8842a-de68-48f2-90ff-ce38392ec359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:44:19.854905 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8842a-de68-48f2-90ff-ce38392ec359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:44:19.855136 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8842a-de68-48f2-90ff-ce38392ec359 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:44:19.856271 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8842a-de68-48f2-90ff-ce38392ec359 admin admin] PATH_INFO: `/v3/users/d1934a2e2edb4b8a93baac501c9c5c3b` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:44:19.857749 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8842a-de68-48f2-90ff-ce38392ec359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1934a2e2edb4b8a93baac501c9c5c3b)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:44:19.860128 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8842a-de68-48f2-90ff-ce38392ec359 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:44:19.907022 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-46c8842a-de68-48f2-90ff-ce38392ec359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1934a2e2edb4b8a93baac501c9c5c3b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:19.913570 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-46c8842a-de68-48f2-90ff-ce38392ec359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1934a2e2edb4b8a93baac501c9c5c3b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:19.914223 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-46c8842a-de68-48f2-90ff-ce38392ec359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1934a2e2edb4b8a93baac501c9c5c3b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:19.922103 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-46c8842a-de68-48f2-90ff-ce38392ec359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1934a2e2edb4b8a93baac501c9c5c3b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:19.922817 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-46c8842a-de68-48f2-90ff-ce38392ec359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1934a2e2edb4b8a93baac501c9c5c3b'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:19.940087 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1416/2881] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:44:19 2022] DELETE /identity/v3/users/d1934a2e2edb4b8a93baac501c9c5c3b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:44:20.115626 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9e500-48ad-472e-8763-808fbad47082 service neutron] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:44:20.125781 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9e500-48ad-472e-8763-808fbad47082 service neutron] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:44:20.182459 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:44:20.183345 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:44:20.183649 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:44:20.183920 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] PATH_INFO: `/v3/projects/84f9c806f7d54963bd886eae2ab22c38` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:44:20.185072 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f9c806f7d54963bd886eae2ab22c38)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:44:20.186951 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:44:20.230143 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f9c806f7d54963bd886eae2ab22c38', 'request_id': 'req-12676adf-478e-4a0b-a98a-3dd1a775e361'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.235687 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f9c806f7d54963bd886eae2ab22c38', 'request_id': 'req-12676adf-478e-4a0b-a98a-3dd1a775e361'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.243751 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f9c806f7d54963bd886eae2ab22c38', 'request_id': 'req-12676adf-478e-4a0b-a98a-3dd1a775e361'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.255219 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] The token cache is being invalidate because project 84f9c806f7d54963bd886eae2ab22c38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:44:20.255531 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.256161 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.257699 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1466/2882] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:44:20 2022] DELETE /identity/v3/projects/84f9c806f7d54963bd886eae2ab22c38 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:44:20.262176 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8842a-de68-48f2-90ff-ce38392ec359 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:44:20.263675 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-46c8842a-de68-48f2-90ff-ce38392ec359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:44:20.276875 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8842a-de68-48f2-90ff-ce38392ec359 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:44:20.333642 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7954d-9294-4a9d-8f06-1141d354ec09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:44:20.334530 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7954d-9294-4a9d-8f06-1141d354ec09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:44:20.334868 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7954d-9294-4a9d-8f06-1141d354ec09 admin admin] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:44:20.335422 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7954d-9294-4a9d-8f06-1141d354ec09 admin admin] PATH_INFO: `/v3/users/7cacb076763149cfab7464b1fbbe365d` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:44:20.336918 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7954d-9294-4a9d-8f06-1141d354ec09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cacb076763149cfab7464b1fbbe365d)` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:44:20.339811 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7954d-9294-4a9d-8f06-1141d354ec09 admin admin] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:44:20.386265 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-34e7954d-9294-4a9d-8f06-1141d354ec09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cacb076763149cfab7464b1fbbe365d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.393334 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-34e7954d-9294-4a9d-8f06-1141d354ec09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cacb076763149cfab7464b1fbbe365d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.393983 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-34e7954d-9294-4a9d-8f06-1141d354ec09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cacb076763149cfab7464b1fbbe365d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.400513 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-34e7954d-9294-4a9d-8f06-1141d354ec09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cacb076763149cfab7464b1fbbe365d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.401216 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.notifications [None req-34e7954d-9294-4a9d-8f06-1141d354ec09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cacb076763149cfab7464b1fbbe365d'} {{(pid=79825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.417269 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1417/2883] 10.222.0.25 () {58 vars in 1223 bytes} [Mon May 9 18:44:20 2022] DELETE /identity/v3/users/7cacb076763149cfab7464b1fbbe365d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 18:44:20.581566 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] Authenticating user token {{(pid=79826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:44:20.583095 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.fernet_utils [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:44:20.596181 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676adf-478e-4a0b-a98a-3dd1a775e361 admin admin] Validating token access rules against request {{(pid=79826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:44:20.656357 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432c60c-095f-4bb1-8499-b7d48b9faed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a379c334e6cd446c80b18fda6ce9162e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '969d4448da94407dbcabadb61787000d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:44:20.657153 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432c60c-095f-4bb1-8499-b7d48b9faed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:44:20.657436 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432c60c-095f-4bb1-8499-b7d48b9faed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:44:20.657717 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432c60c-095f-4bb1-8499-b7d48b9faed6 admin admin] PATH_INFO: `/v3/projects/227044f3d2f94cea9da8413a52f95829` {{(pid=79826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:44:20.658877 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0432c60c-095f-4bb1-8499-b7d48b9faed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=227044f3d2f94cea9da8413a52f95829)` {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:44:20.660809 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0432c60c-095f-4bb1-8499-b7d48b9faed6 admin admin] RBAC: Authorization granted {{(pid=79826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:44:20.705212 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-0432c60c-095f-4bb1-8499-b7d48b9faed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '227044f3d2f94cea9da8413a52f95829', 'request_id': 'req-0432c60c-095f-4bb1-8499-b7d48b9faed6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.710412 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-0432c60c-095f-4bb1-8499-b7d48b9faed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '227044f3d2f94cea9da8413a52f95829', 'request_id': 'req-0432c60c-095f-4bb1-8499-b7d48b9faed6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.718283 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-0432c60c-095f-4bb1-8499-b7d48b9faed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '227044f3d2f94cea9da8413a52f95829', 'request_id': 'req-0432c60c-095f-4bb1-8499-b7d48b9faed6'} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.728888 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-0432c60c-095f-4bb1-8499-b7d48b9faed6 admin admin] The token cache is being invalidate because project 227044f3d2f94cea9da8413a52f95829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79826) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 18:44:20.729175 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-0432c60c-095f-4bb1-8499-b7d48b9faed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.729796 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: DEBUG keystone.notifications [None req-0432c60c-095f-4bb1-8499-b7d48b9faed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 18:44:20.731381 dv-cinder-841125-3-iscsi devstack@keystone.service[79826]: [pid: 79826|app: 0|req: 1467/2884] 10.222.0.25 () {58 vars in 1232 bytes} [Mon May 9 18:44:20 2022] DELETE /identity/v3/projects/227044f3d2f94cea9da8413a52f95829 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 18:48:54.346520 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7954d-9294-4a9d-8f06-1141d354ec09 admin admin] Authenticating user token {{(pid=79825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 18:48:54.347906 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-34e7954d-9294-4a9d-8f06-1141d354ec09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:48:54.415043 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7954d-9294-4a9d-8f06-1141d354ec09 admin admin] Validating token access rules against request {{(pid=79825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 18:48:54.469944 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0676f8-96ad-436b-a00c-b25856b0be78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f747dcb955fa499a972c46587d3db294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4849b58351c74caf92a584686c74e397', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 18:48:54.470586 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0676f8-96ad-436b-a00c-b25856b0be78 service placement] REQUEST_METHOD: `GET` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 18:48:54.470865 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0676f8-96ad-436b-a00c-b25856b0be78 service placement] SCRIPT_NAME: `/identity` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 18:48:54.471139 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0676f8-96ad-436b-a00c-b25856b0be78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 18:48:54.472891 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.fernet_utils [None req-9e0676f8-96ad-436b-a00c-b25856b0be78 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 18:48:54.483543 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0676f8-96ad-436b-a00c-b25856b0be78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 18:48:54.485086 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0676f8-96ad-436b-a00c-b25856b0be78 service placement] RBAC: Authorization granted {{(pid=79825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 18:48:54.494549 dv-cinder-841125-3-iscsi devstack@keystone.service[79825]: [pid: 79825|app: 0|req: 1418/2885] 10.222.0.25 () {60 vars in 1365 bytes} [Mon May 9 18:48:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)